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METHODS OF INTERACTING WITH AUDIO 
AND AMBIENT MUSIC 

RELATED APPLICATION DATA 

Iliis application claims the benefit of application 60/134, 
782, filed May 19, 1999, attached hereto as Appendix A. 

The technology detailed in the present application is also 
related to that detailed in application Ser. No. 09/343,104, 
filed Jun. 29, 1999; Sen No. 09/292,569, filed Apr. 15, 1999 
(now abandoned in favor of continuation application Ser. No. 
10/379,393, filed Mar. 3, 2003, now Pat. No. 7,263,203); Ser. 
No. 09/3 1 4,648. filedMay 1 9, 1 999 (now U.S. Pat. No. 6,68 1 , 
028); 60/141,763, filed Jun. 30, 1999; 60/158,015, filed Oct 
6, 1999; 60/163,332, filed Nov. 3, 1999; 60/164,619, filed 
Nov. 10, 1999; Ser, No. 09/452,023, filedNov, 30, 1999(now 
U.S. Pat. No. 6,408,082); 09/452,021, filed Nov. 30, 1999; 
and in U.S. Pat. No. 5,862,260. 

INTRODUCTION 

1 6 year old Bob struts into the coffee shop down fi:om high 
school with a couple of buddies, a subtle deep pound in the 
ambient sound track lets them know they 're in the right place. 
The three of them instinctually pull out of their pockets their 
audio Birddawgs (a small hand held unit about the size and 
style of an auto-door-alann device, or "fob"), and when they 
see the tiny green light, they smile, high five, and push the big 
"GoFetch" button in synchrony. That tune will now be wait- 
ing for them at home, safely part of their preferred collection 
and ever- so-thankfiilly not lost to their collective bad memory 
(if they even knew the name of the artist and tune title in the 
first place!). 

33 year old Mary is at home listening to the latest batch of 
hohday tunes being offered up over her 2-decade-long favor- 
ite radio station. She's spent many days now half-consciously 
culling the tunes for that perfect arrangement for the new 
year's bash that she regrettably agreed to host. 10:40 AM rolls 
around and some new tune catches her ear, a tune she knows 
can work well following the jingle-cats rendition of Straw- 
berry Fields. She half jogs over to the stereo and hits the 
"GoFetch" button. In a few days, she'll sit down at the com- 
puter and put together the fial sound track for the gala 
evening ahead, her play list dutifiilly waiting for her shufifting 
instructions and desired start time. 

49 year old Jack (the financial analyst) is thoroughly bored 
sitting in the crowded gate D23 at Dulles. Droning 20 feet up 
and over his head is the airport network station, currently 
broadcasting the national weather report. As the segue to the 
business segment approaches, the teaser review mentions that 
they'll be having a report on today's rally in the bond market 
and the driving forces behind it. Jack pulls out his Birddawg- 
enabled Palm Pilot on the off-chance they actually will have 
a little depth in the reporting. Indeed, as the segment plays and 
starts discussing the convoluted effects of Greenspan's 
speech to the Memet-B-Free society, he taps the "GoFetch" 
button, knowing that once he gets back to his main browsing 
environment he will be able to follow dozens of links that the 
airport network has pre-assigned to the segment. 

The foregoing and other features and advantages of the 
present invention will be more readily apparent fi-om the 
following detailed desoiption, which proceeds with refer- 
ence to the accompanying figures. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagram showing the participants, and chaimels, 
involved in the distribution of music. 

FIG. 2 shows a conceptual model of how music artists, 
record labels, and E-Music distributors can all interact with a 


Media Asset Management System, of which several are 
detailed in the following specification. 

FIG. 3 is a block diagram of a device according to one 
embodiment o f the present invention. 
5 FIG. 4 is a block diagram of a system in which the device 
of FIG. 3 may be utilized. 

DETAILED DESCRIPTION 

1 0 Referring to FIG. 3, a device 10 according to one embodi- 
ment of the present invention includes a microphone 12, an 
A/D converter 13, a processor 14, one or more indicators 16, 
one or more buttons 18, a wireless interface 20, and a power 
source 22. 

15 The device can be packaged in a smaU plastic housing, 
preferably as small as is practical (e.g., sized and configured 
to serve as a key chain ornament, perhaps akin to the Toma- 
gatchi toys that were recently popular). The housing has one 
or more small holes to permit audio penetration through the 

20 housing to the microphone 12. 

The processor 14 can take various forms, including a dedi- 
cated hardware device (e.g., an ASIC), a general purpose 
processor programmed in accordance with instructions stored 
in non-volatile RAM memory, etc. 

25 The indicators 16 can be as simple as a single LED lamp, or 
as complex as an alphanumeric LCD or other multi- element 
display. In one embodiment, the indicator simply indicates 
when the processor has decoded a watermark in audio sensed 
by the microphone. More elaborate signahng techniques can 

30 of course be used, including two- or three-color LEDs that 
can be used to signal different states with different colors, 
indicators with flashing patterns or changing displays, etc. 

The buttons 18 are used by the user to indicate an interest in 
the audio jxist-heard. In one embodiment, there is a single 

35 button 18, and it is emblazoned with a stylized legend that can 
serve as a trademark or service mark, e.g., GetIt!, GoFetch, 
Birddawg, something Batman-esque ("Wham," "Zappp" 
"Pow! !," etc.), or something more mundane (e.g.. Capture). 
The power source 22 can be a battery, solar cell, storage 

40 capacitor, or other source of energy suitable for powering the 
components of the device 10. 

The wireless interfece 20 serves to exchange data with a 
relay station 24 (FIG. 4). In one embodiment, the interface is 
radio-based, and provides a one-way commimications chan- 

45 nel. In other embodiments other wireless technologies can be 
used (e.g., IR), and/or two-way communication can be pro- 
vided. 

The relay station can be a cellular repeater (if the interface 
transmits using cellular frequencies and protocols), or a local 

50 receiver, e.g., associated with the user's computer. The relay 
station can also be a paging system relay station (e.g., as are 
used for two-way pagers), or may be a low earth orbit satel- 
hte-based repeater. 
In operation, the processor monitors the ambient audio for 

55 the presence of encoded data, e.g., a digital watermark, and 
decodes same. If power considerations permit, the device is 
"always-on." In olier embodiments, one of the buttons 18 can 
be used to awaken the device. In such other embodiments, 
another button-press can serve to turn-off the device, or the 

60 device can power-down after a predetermined period, e.g., of 
not sensing any watermarked audio. 

A mmiber of techniques for watermarking audio (and 
decoding same) are known, as illustrated by U.S. Pat. Nos. 
5,862,260, 5,963,909, 5,940.429, 5,940,135. 5.937,000, 

65 5.889,868, 5,833.432, 5,945,932, W09939344 (correspond- 
ing to U.S. application Ser. No. 09/017,145. now U.S. Pat. 
No. 6,145,081), and W09853565 (corresponding to US, 
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application Ser. Nos. 08/858,562 and Ser. No. 08/974,920, In some embodiments, the repository 30 (which may be 
DOW U.S. Pat. Nos. 5,940,135 and 6,175,627, respectively). co-located with server 28, or not) includes vanous data 
Commercially-available audio watermarking software beyond the audio itself. For example, the repository can store 
includes that available from AudioTrack, Verance (formerly a collection of metadata (e.g., XML tags) corresponding with 
Aris/Solana), Cognicity, Liquid Audio, and others. 5 each stored item of audio. This metadata can be transmitted to 
Thedatapayloadencodedbythewatennark(theaudio-rD) the user's destination 32, or can be used, e.g., for rights 
may take various forms. One is a Digital Object Identifier- management purposes (to Imnt the user s reproduction or 
an ID corresponding to the standardized digital object num- re-distribution nghts for the audio, etc.), to estabhsh a fee for 
bering system promulgated by the Memational DOI Foun- the audio, etc. One smtable metatag standard is that under 
dation (www.doi.oiB). Another is to include plural data fields development by <mdecs> (Interoperability of Data m 
variously representing, e.g., the name of the publisher, the E-Commerce Systems, www.mdecs.org). 
name of the artist, the title of the work, the date of publication, The audio data can be delivered in streaming form, such as 
etc., etc. Another is to encode a unique identifier (UID), e.g., using technology available from RealNetworks (RealAudio), 
of 16-64 bits. The UID serves as anindexto a remote database Microsoft (\\^mdows Media Player), MP3, Audiobase, Beat- 
where additional information (e.g., pubUsher, artist, title, date 15 nik, Bluestreak.com, etc. The former three systems require 
of publication, etc., are stored). The data transmitted from the large (e.g., megabytes) player software on the receiving (cli- 
device 10 to the relay station 24 typically includes some or all ent) computer; the latter do not but instead rely, e.g., on small 
of the watermark payload data, and also includes data iden- Java applets that can be downloaded with the music, 
tifying the de\'ice 10, or its user (user- ID data). Again, this Alternatively, the audio can be delivered ma file format. In 
data can include several data fields (e.g. user name, audio 20 some embodiments the file itself is delivered to the user's 
delivery information such as email address or URL, age, destination 32 (e.g., as an email attachment). In others, the 
gender, model of device 10, etc.). Alternatively, a serial num- user is provided a URL to permit access to, or downloading 
ber or other unique identifier can be used, which serves as an of, the audio. (The URL may be a web site that provides an 
index to a database have a corresponding record of informa- interface through which the user can pay for the requested 
tion relating to the user and/or device. 25 music, if pre-payment hasn't been arranged.) 

The audio-ID and user-ID data are typicaUy formatted and The user's destination 32 is typically the user's own com- 
encoded by the device 10 according to a protocol that pro- puter. If a "Uve" IP address is known for that computer (e.g., 
vides error correcting, framing, and other data useful in assur- by reference to a user profile database record stored on the 
ing reliable transmission to the relay station, and/or for fur- server 28), the music can be transferred immediately. If the 
ther transport user's computer is only occasionally connected to the inter- 
Some embodiments of device 10 recognize just a single net. the music can be stored at a web site (e.g. protec^^^ 
form of watermarking, and can miderstand only payload data ^ ^^er-set password), and can be downloaded to the user s 
presented in a single format. In other embodiments, the computer whenever it is convement. 
device may be capable of recognizmg watermarking accord- ^ embodiments, the destmatioii 32 is a personal 
ing to several different techniques, and with several different music library associated with the user. The hbrary can take the 
payload formats. This latter flmctionaUty can be achieved, form, e.g., of a hard-disk or semiconductor memory array m 
e.g., by cyclically trying different decoding techniques until w^ch the user customanly stores music. This storage device 
one that produces valid output data (e.g., by reference to a is adapted to provide music data to one or more playback umts 
checksum or otiier indicia) is obtained. That decoding tech- employed by the user (e.g. a personal MP3 player, a home 
nique and payload interpretation can thereafter be used until stereo system, a car stereo system, etc.). In most mstaDattons, 
valid output data is no longer obtained, the library is physicaUy located at the user's residence, but 
In some embodiments, the device 10 transmits data to the remotely sited, f^^^^^^^o^^^^^ ^^''^ 
relay station at the moment the user presses the button 18. In ^^^ranes of many other users at a central location 
other embodiments, a store- and- forward mode is used. Tliat 45 P^^f^^^ ""^l'^ ^'^"^ T T "^^^"^S^T' 
is, when the user presses the button 18, the decoded water- ^ectton. Or it can be eqmpped wi^ wireless capabihUes, 
mark data is stored in memory within the device, mreafter, permittmg it to receive digital music from wireless broaden 
e.g., when the device is coupled with a "nest" or "holster" at (^-8- a transmitter associated with tiie server 28) In 
the user's computer (or when download capabiUty is other- either case, the hbrary can provide music to the user s play- 
wise activated), the stored data is downloaded^ither 50 back devices by short-range wneless broadcast 
through that device or otherwise. ^ rnanv embodmients, technology such as that available 
• ^ ^ .uj - tn r from Sonicbox, pennits audio data deUvered to the computer 
The mfrastructure between the device 10 and dehvery of ""r^ ^^i^^ukj , f/^ 
J x 1^ * J ^- 4. 1 to be short range FM-broadcast by the user s computer to 
the audio to its ultunate destmation can take mynad forms. " ^1 j- • • a .r™4^,rr. 
OneisshowninFIG.4.Inthisarrangement,someoraUofthe neari^y FM radios usmg othenvi^^^^ 
data received by the relay station 24 is routed through the 55 Some implemoitations of the present mvention sup^rt 
internet 26 to a server 28. (The server 28 can be a "Medi- several different dehvery technologies (e.g., sti-eammg, fi^e, 
aBridge" server of the type described, e.g., in the assignee's URL), and select among them m accordance with the profiles 
application 60/164,619, filed Nov. 10, 1999, and Ser. No. of different users. 

09/343,104, filed Jun. 29, 1999.) Server 28 parses the data and Payment for the audio (ifneeded) can be accomplished by 

routes some or all of it to a data repository 30 at which the 60 numerous means. One is by charging of a credit card account 

audio requested by the user is stored. This repository, in ttim, associated with the user (e.g., in a database record corre- 

dispatches the audio to the user (e.g., to a compute, media spending to the user-ID). 

player, storage device, etc.), again through the internet. (Ad- Some implementations of the invention make use of secure 
dress information detailing the destination 32 of the audio delivery mechanisms, such as those provided by InterTmst, 
may be included in the data sent from the device 10, or can be 65 Preview Systems, etc. In addition to providing, secure con- 
retrieved from a database at the server 28 based on a user- ID tainers by which the audio is distributed, such systems also 
sent from the device 10.) include their own secure payment fecilities. 
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By such arrangements, a user can conveniently compile an 
archive of favorite music — even while away from home. 

To provide a comprehensive disclosure without imduly 
lengthening this specification, the disclosures of the applica- 
tions and patents cited above are incorporated herein by ref- 
erence. 

Having described and illustrated the principles of my 
invention with reference to a preferred embodiment and sev- 
eral variations thereof, it should be apparent that the detailed 
embodiment is illustrative only and should not be taken as 
Umiting the scope of my invention. 

For example, while the invention is illustrated with refer- 
ence to a button that is activated by the user to initiate capture 
of an audio selection, other interfaces can be used. For 
example, in some embodiments it can be a voice-recognition 
system that responds to spoken commands, such as "capture" 
or "record." Or it can be a form of gesture interface. 

Likewise, while the invention is illustrated with reference 
to a stand-alone device, the same functionahty can be built- 
tnto radios (including internet-based radios that receive wire- 
less IP broadcasts), computer audio systems, and other appli- 
ances. In such case the microphone can be omitted and, in 
some cases, the wireless interface as well. (The data output 
from the device can be conveyed, e.g., through the network 
connection of an associated computer, etc.) 

Moreover, while the invention is illustrated with reference 
to an embodiment in which audio, alone, is provided to the 
user, this need not be the case. As in the DuUes airport sce- 
nario in the introduction, the server 28 can provide to the user 
several internet Unks associated with the sensed audio. Some 
of these links can provide commerce opportunities (e.g., to 
purchase a CD on which the sensed audio is recorded). Others 
can direct the user to news sites, concert schedules, fan-club 
info, etc. In some such embodiments, the ancillary informa- 
tion is provided to the user without the audio itself 

Although not particularly detailed, the data provided to the 
user's destination typically includes information about the 
context in which the data was requested. In a simple case this 
can be the time and date on which the user pressed the Capture 
button. Other context information can be the identification of 
other Birddawg devices 10 that were nearby when the Cap- 
ture button was pressed. (Such information can be gleaneil, 
e.g., by each device transmitting a brief Who Ami message 
periodically, receiving such messages from other nearby 
devices, and logging the data thus received.) 

Still other context information might be the location from 
which the Capture operation was initiated. This can be 
achieved by decoding of a second watermark signal, e.g., on 
a low level white-noise broadcast. The public address system 
in pubhc places, for example, can broadcast a generally- 
indiscemable noise signal that encodes a watermark signal. 
Devices 10 can be arranged to detect two (or more) water- 
marks from the same audio stream, e.g., by reference to two 
pseudo-random sequences with which the difiFerent water- 
marks are encoded One identifies the audible audio, the other 
identifies the location. By such an arrangement, for example, 
the device 10 can indicate to the server 28 (and thence to the 
user destination 32) the location at which the user encoun- 
tered the audio. (This notion of providing location context 
information by subliminal audio that identifies the location 
has powerful appUcations beyond the particular scenario con- 
templated herein.) 

In some embodiments, the device 10 can buffer watermark 
information from several previous audio events, permitting 
the user to scroll back and select (e.g., in conjunction with a 
screen display 16) the ID of the desired audio. 
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An arrangement like the foregoing may require that the 
decoded watermark information be interpreted for the user, so 
that the user is not presented simply a raw binary watermark 
payload. The interpreted information presented to the user 
5 can comprise, e.g., the source (CNN Airport News, WABC 
Radio, CD-ROM, MTV), the artist (Celine Dion), the title 
(That's the Way It Is), and/or the time decoded (3:38 :02 p.m.), 
etc. 

One way to achieve the foregoing functionahty is to convey 

10 both the binary UID payload and abbreviated text (e.g., 5- or 
6-bit encoded) through the watermark "channel" on the 
audio. In one such arrangement, the watermark channel con- 
veys data a UID, four characters of text, and associated error- 
correcting bits, every ten seconds. In the following ten sec- 

15 onds the same UID is conveyed, together with the next four 
characters of text. 

Another way to achieve such functionahty is to provide a 
memory in the device 10 that associates the watermark pay- 
load (whether UID or field-based) with corresponding textual 

20 data (e.g., the source/artist/title referenced above). A 1 mega- 
byte semiconductor non- volatile RAM memory, for example, 
can serve as a look-up table, matching code numbers to artist 
names and song titles. When the user queries the device to 
leam the identify of a song (e.g., by operating a button 18), the 

25 memory is indexed in accordance with one or more fields 
from the decoded watermark, and the resulting textual data 
from the memory (e.g. source/artist/title) is presented to the 
user. 

Such a memory will commonly require periodic updating. 

30 The wireless interface 20 in device 10 can include reception 
capabiUties, providing a ready mechanism for providing such 
updated data. In one embodunent, the device "awakens" 
briefly at otherwise idle moments and tunes to a predeter- 
mined frequency at which updated data for the memory is 

35 broadcast, either in a baseband broadcast channel, or in an 
ancillary (e.g. SCA) channel. 

In variants of the foregoing, internet deUvery of update data 
for the memory can be substituted for wireless delivery. For 
example, a source/artist/title memory in the device 10 can be 

40 updated by placing the device in a "nest" every evening. The 
nest (which may be integrated with a battery charger for the 
apphance) can have an internet connection, and can exchange 
data with the device by infirared, inductive, or other proxim- 
ity-coupling technologies, or through metal contacts. Each 

4 5 evening, the nest can receive an updated collection of source/ 
artist/title data, and can re-write the memory in the device 
accordingly. By such arrangement, the watermark data can 
always be properly interpreted for presentation to the user. 
The "Capture" concepts noted above can be extended to 

50 other functions as well. One is akin to forwardmg of email. If 
a consumer hears a song that another fiiend would enjoy, the 
hstener may send a copy of the song to the fiiend. This 
instruction can be issued by pressing a "Send" button, or by 
invoking a similar fimctioii on a graphical (or voice- or ges- 

55 ture-responsive) user interface. In response, the device so- 
instructed can query the person as to the recipient. The person 
can designate the desired recipient(s) by scrolling through a 
pre-stored list of recipients to select the desired one. (The list 
can be entered through a computer to which the device is 

60 coupled) Alternatively, the user can type-in a name (if the 
device provides a keypad), or a portion thereof sufiicient to 
uniquely identify the recipient. Or the person may speak the 
recipient's name. As is conventional with hands- free vehicle 
cell phones, a voice recognition unit can listen to the spoken 

65 instructions and identify the desired recipient. An "address 
book"- like feature has the requisite information for the recipi- 
ent (e.g., the web site, IP address, or other data identifying the 
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location to which music for that recipient should stored or 
queued, the format in which the music should be delivered, 
etc.) stored therein. In response to such command, the appli- 
ance dispatches instructions to the server 28, including an 
authorization to incur any necessary charges (e.g., by debitmg 
the sender's credit card). Again, the server 28 attends to 
delivery of the music in a desired manner to the specified 
recipient. 

Still further, a listener may query the device (by voice, GUI 
or physical button, textual, gesture, or other input) to identify 
CDs on which the ambient audio is recorded. Or the listener 
may query the device for the then-playing artist's concert 
schedule. Again, the appliance can contact a remote database 
and relay the query, together with the user ID and audio ID 
data. The database locates the requested data, and presents 
same to the user— either through a UI on device 10, or to the 
destination 32. If desired, the user can continue the dialog 
witha further instruction, e.g., to buy one ofthe CDs on which 
the then-playing song is included. Again, this instruction may 
be entered by voice, GUI, etc., and dispatched from the device 
to the server, which can then complete the transaction in 
accordance with pre-stored information (e.g. credit card 
account number, maihng address, etc.). A confir m i n g mes- 
sage can be relayed to the device 10 or destination 32 for 
presentation to the user. 

While the invention particularly contemplates audio, the 
principles detailed above find applications in many other 
media, and in many other appUcations ofthe MediaBridge 
server 28. 

Moreover, while the invention particularly contemplates 
watermarks as the channel by which audio is identified, in 
other embodiments different techniques can be used. For 
example, digital radio protocols provide ID fields by which 
audio can be identified. Similarly, IP protocols for internet 
delivery of radio include identification fields within their 
packet formats. Accordingly, audio distributed according to 
formats that include audio IDs thereiii can Ukewise be 
employed according to the present invention. 

In view ofthe many embodiments to which the principles 
of our invention may be apphed, it should be apparent that the 
detailed embodiment is illustrative only and should not be 
taken as limiting the scope ofthe invention. Rather, ^ye claim 
as our invention all such modifications as may fall within the 
scope and spirit of the following claims, and equivalents 
thereto. 

APPENDIX A 

Application 60/134,782, filed May 19, 1999 

METHODS AKD SYSTEMS EMPLOYING 
DIGITAL WATERMARKING 

FIELD OF THE INVENTION 

The present invention relates to appUcations of digital 
watermarking in conjimction with audio, video, imagery, and 
other media content. 
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BACKGROUND 

Watermarking (or "digital watermarking") is a quickly 
growing field of endeavor, with several differait approaches. 
The present assignee's work is reflected in U.S. Pat. Nos. 
5 841,978, 5,768,426, 5,748,783. 5,748,763, 5,745,604, 
5*7 10,834, 5,636,292, 5,721,788, and laid-open PCT appU- 
cations WO 97/43736 and WO 99/10837. Other work is lUus- 


trated by U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659,726, 
5,664,018, 5,671,277, 5,687,191, 5,687.236, 15 5,689,587, 
5,568,570, 5,572,247, 5,574,962, 5,579,124, 5,581,500, 
5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664, 
5,530,759,5,539,735, 4,943,973, 5,337,361, 5,404,160, 
5,404,377, 5,315,098, 5,319,735, 5,337,362, 4,972,471, 
5 161,210, 5,243,423, 5,091,966, 5,113,437,4,939,515, 
5,374,976, 4,855,827, 4,876,617, 4,939,515, 4 963 9^^ 
4 969 04 1 , and published foreign appUcations WO 98/02864, 
EP 822,550, WO 97/39410, WO 96/36163, GB 202,196,167, 
EP 777,197, EP 736,860, EP 705,025, EP 766,468, EP 782, 
322 WO 95/20291, WO 96/26494, WO 96/36935, WO 
96/42151, WO 97/22206, WO 97/26733. 

Most ofthe work in watermarking, however, is not in the 
patent Uteiature but rather in pubUshed research. In addition 
to the patentees of the foregoing patents, some of the other 
workers in this field (whose watermark-related wntmgs can 
by found by an author search in the INSPEC database) 
include L Pitas, Eckhard Koch, Jian Zhao, Nonshige 
Morimoto, Laurence Boney, Kineo Matsui, A.Z. Tirkel, Fred 
Mintzer, B. Macq, Ahmed H, Tewfik, Frederic Jordan, Nao- 
hisa Komatsu, and Lawrence 0 'Gorman. 

The artisan is assumed to be famiUar with the foregoing 
prior ait. 

In the present disclosure it should be understood that ref- 
erences to watermarking encompass not only the assignee's 
watermarking technology, but can Ukewise be practiced with 
any otiier watermarking technology, such as those mdicated 
above. 

Watermarking has various uses, but the present specifica- 
tion details several new uses that provide functionality and 
features not previously available. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagram showii^ the participants, and channels, 
involved in the distribution of music. 

FIG. 2 shows a conceptual model of how music artists, 
record labels, and E-Music distributors can all interact with a 
Media Asset Management System, of which several are 
detailed in the following specification. 

DETfiJLED DESCRIPTION 

For expository convenience, much of the following discus- 
sion focuses on music, but the same principles and techniques 
are largely or whoUy appUcable to other source data, whether 
non-music audio, video, stiU imagery, printed materials, etc. 

Music Asset Management 

Referring to the figures, the music distribution process 
begins with a creative artist 10. The artist's music has tradi- 
tionaUy been distributed by a record label 12. (While tiie 
55 following discussion, refers to distribution through such a 
label, it should be understood that such distribution can just as 
well be effected directed under tiie artist's control, without a 
record label intermediary.) 
In traditional distribution 14, the record label produces 
60 tangible media, such as records, tapes, videos (e.g. music 
videos), and CDs 16. These media are physically distnbuted 
to end-consumers 18. AdditionaUy, the label 12 distributes the 
music media to outlets 20, such as radio and TV stations, 
cable and sateUite systems, etc., whidi broadcast (or narrow- 
65 cast) the artist's work to an audience. Distribution through 
such media outiets may be monitored by playout trackmg 
services. Playout f acking data, coUected by firms mcludmg 
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* u w XT Air AP wiUT ptr he used to com- a voluntary specification that wiU assure proper compensa- 

"l:Ssingly, the distribution of the music to the media {o-Newsw^^^^^^ 

outlets is perfonned electronicaUy. Such distribution first 5 Industry Eyes Web, Dec. 31, 1998.) 

took the form of analog audio over high quality landlines or Yet another mitiative has been announced by Solana and 

la?eime ch^^^^^^^ audio quickly supplanted analog ASCAP. Other companies promotmg watennarkmg for 

Tudlo S SSriStion channels due I higher fidelity. music include Aris Technology. MCY.com. and AudioSoft. 

More recentiy distribution of the music from the record The watermark payload can represent vanous types ot 

labelstothemediaoutletshasoccurredoversecureUnks,now lo data. An exemplary payload includes data relatmg to me 

includino the internet. Such security was first provided sim- artist, distribution entity, tide, and copyright date/propnetor. 

ply by scrambhng the audio signal or data. More sophisti- Additionally, the payload can mclude a digital object idenu- 

cated "containef'-based systems are now coming into vogue. fier-an ISBN-like number issued by a central organization 

in which the audio is "packaged" (often in encrypted fonn) (e.g, a rights management orgamzation) to umquely identity 

with ancillary data. the work. 

Electronic distribution of music to the consumer is also guch payload data can be encoded literally (e.g. tne utie oy 

aaining popularity, presenUy in the MP3 format primarily. a series of ASCH characters, etc.). In other embodimaits, 

The music providers may deal directly with the pubUc, but ^odes or abbreviations can be employed with each code bav- 

more commonly effect such consumer distribution through a j^g ^ known meaning. In still other embodiments, the data can 
newly emerging tier ofdigital media outlets, such as internet 20 t»e meaningless by itself, but may serve as a key (e.g., a 

sites that specialize inmusic. From such sites, consumers can Unique Identifier, or UID) into a remote data database or 

download digital audio files into personal digital audio play- repository. An example of such a remote data repository is a 

ers. (The Diamond Rio, and the Audible MobilePlayer web site at a Master Global Address (MG A) associated with 

devices are some of the first of what will doubtless be a large content, as detailed below. 

number ofentrants into this personal internet audio appUance 25 ^ exemplary data payload may, for example, have the 

market.) Or the downloaded data can be stored by the con- following format: 
sumer-recipient onto any other writeable media (e.g. hard 

disk. CD. DVD, tape, videotape, etc.). Typically a personal ABCDEFGHI 
computer is used for such downloading, but this intermediary 

may be dispensed with by coupUng next generation of per- 3o ^^g^e A is a six-byte (8-bits to a byte) ASCn string sendng 

sonal audio appliances to an internet-like link. ^ ^g-^^ ^^jg^t identifier (which may serve as a link to a 

Ihe data downloaded by the consumer can be stored either Master*' Global Address through a defeult name server, as 

in the native digital format, translated into another digital discussed below), Bis a two-byte ASCH field serving as a key 

format (which translation may include decryption), converted ^ "artist" field of the remote database, C is a three-byte 
into analog and recorded in analog form, etc. 35 ^g^jj f^^j^ serving as a key into a 'litle" field of the remote 

Unauthorized copymg or use of the music can occur any- database; D is a 1 4-bit field serving as a key into a "label" field 

where in die foregoing channels. However, one of the greatest remote database, E is an 8-bit integer representing the 

risks occurs once the music has been delivered to the con- ^^^y^^^ year of first pubhcation (with 0 representing the year 

sumer (whether by tangible media, by traditional broadcast ^ooO)* F is a 10-bit field serving as a key into a "price" field of 
media outlets, by emerging digital distribution, or otherwise). 40 ^^^^^^ database, G is a two-byte usage control string 

The general idea of embedding auxiliary data mto music (detailed below), H is a streaming data channel, and I is a 

(i.e. watermarking) has been widely proposed, but so far has ^^^.^ serving as a cyclic redundancy checksum for the 

been of limited appUcability. foregoing. (More sophisticated error correcting checksums 

For example, GoodNoise is planning to embed a digital of course, be employed.) This payload format totals 1 36 

signature -- termed a multimedia identifier, or MMI - m its 45 ^^^.^^51^^ crC coding and the streaming data 

MP3 music. MMI will register the song and its author with a channel 

Ucensing number. In addition to providing information about This payload is encoded repeatedly, or redundantly 

the songwriter and distributor, this digital encoding may also ^^^.^ ^ payload can be decoded 

include lyrics, finer notes, an^ other ^^^^^f^^^^^^^ from partial excerpts of the music. 

the proposed uses serve only to convey 50 ^ . ■ ^^^^ ^^^i^^ly perceptuaUy adaptive, so 

distributor to the consumer; use for "^^clong is acfively 

disclaimed.(WiredNews,"GoodNoiseTagsMP3Files,"Feb. {Jjg^^.^^^^^ 

^'^^^^■^ - ^ f ^ w r encoding and vice versa. Various techniques for perceptually 

Ihe Genuine Music Coahtion- a partnership of vanous ^nown For example, some tie the 

companiesinthemusicdistributionbusiness-h^^^^^ 53 

announcedplanstoemploywatennarkingof^^ Some^^^^ 

watermarking technology, to be provided by Liqmd Audio ™^ spectrally or temiorally-adjoining signal 

will convey data specifying the artist or Foduca|^« S iSgto^eJ y sS other approaches fill gaps in die 

copyrightdata,andanumbertotrackown^^^^^^^^ Isic's^S 

tion hopes that die provision of tins embedded mformation 60 ^^^^ ^^^^^^ ^^e patents incorporated by refer- 

wiU help tiiwart piracy. Industry observers beheve Liqmd techmques are aeiauca u pa 

Audio will next introduce playback technology only plays ^"^f ■ ^u^.:^p«ts t^ceDtuallv adaptive encoding is 

IndustryAssockuonofAxnenca(RIAA).TcnnedtheS Somp^aSw^Wa^^o'^^^^^ 
Digital Music Initiative (SDMI), the program sedcs to define teredtoempiiasizeiowircqucui^y i~ 
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than 500 hz), high frequency audio components (e.g. higher 4 — four playbacks permitted 

than 2500 hz), or mid- frequency audio components (500- 5— five playbacks permitted 

2500 hz). 6—1 0 playbacks permitted 

The streaming data field channel (H) is a medium by which 7— imlimited playbacks permitted 

data can be conveyed from a distribution site (or other site) to 5 8 — prefer to associated data (within the watermark, or 

the end user. Such data may be entirely imrelated to the stored at a remote site) which specifies number of per- 

underlying work. For example, it may serve a utilitarian pur- mitted playbacks. 

pose, such as conveying data to a memory in the consumer The playback device may include a non- volatile store in 

device to replace previously- stored data that is out-of-date. It which the number of peraiitted playbacks is stored for each 

may be a commercial channel on which bandwidth is sold for i o track of music. The device would decrement this number at 

access to the consumer or the consumer's device. Essentially the beginning of each playback. 

any purpose can be served by this streaming data field. Unlike The usage control string can also include a two-bit field 

most of the other fields, the streaming data field may not (bits ten and eleven) indicating recording permissions. A 

endlessly repeat the same data, but can convey data that value of 0 means that data corresponding to the MP3 audio 

changes with time. is (regardless of digital format) should never be made available 

Desirably, the encoding is performed in a manner permit- to another digital device. A value of 1 means that the data 

ting recovery of the watermark data even if the audio is corresponding to the MPS data may be made available once to 

corrupted, e.g. by format conversion, re- sampling, tape wow another digital device. A value of 2 means that the data may be 

and flutter, compression, coding, or various forms of audio made available an unlimited number of times to other digital 

processmg (e.g. filtering, pre-emphasis, re-scaling, etc.). One 20 devices. (Value 3 is reserved.) 

way to provide for such robustness is to encode a signal of Another data field that can be included in an audio water- 
known character that can be recognized through all such mar k is a rating that indicates age-appropriateness. Music 
corruption. By identifying such known signal, the watermark violence or sexual themes might be given a rating akin to 
signal can then be decoded. (The known signal can take the MPAA "PG-13" or 'TR" rating. Audio appliances may be 
various forms, e.g. a synchronization signal, a marker signal, 25 programmed to recognize the rating of incoming music, and 
calibration signal, a universal code signal as described m to mterrupt playback if the rating exceeds a certain threshold 
applicant ' s patents , etc .) setting . Various known techniques can be employed to as sure 
In some embodiments, a wateraaark "dial-tone" signal is ^^at such settings cannot readily be changed, e.g., by juvenile 
provided. This dial-tone signal is a low amplitude, relatively listeners. 

wideband, repetitive signal that commonly conveys only lim- 30 Another data field that can be included in an audio water- 

ited information (e.g. a smgle bit of mformation). Its presence ^ ^^^^ f^^i^ can indicate either the date the 

m an audio signal can serve as a "do not record," or similar ^^^^^ watermarked, or a date in the fiiture on which 

mstruction signal. Alternatively, or m addition, the dial-tone certain rights associated with the music should change. Some 

signal can serve as an aid m "locking" to a plural-bit digital consumers, for example may not wish to purchase perpetual 

watermark signal that is also encoded m the audio. For 35 playback rights to certain musical selections. The right to play 

example, the cyclical repetition of the signal can serve to ^ selection for 6 months may suffice for many consumers, 

idenufy tiie start of the plural-bit digital watermark signal. Or especially if the price is discounted in viewof the limited term, 

the spectrum or repetition rate of the signal can identify any Suchanarrangement would not be whoUy disadvantageous to 

temporal corruption of the audio. An exemplary such signal is music distributors, since some consumers may end up pur- 

detailed as a "simple umversal code" m Pat. No. 5,636,292. 40 ^i^^^^g ^usic twice if their initial assessment of a musical 

A track of music can be pre-authonzed for specified types selection's appeal was too short-sighted. (Naturally, the play- 

of use. For example, the usage control string of the watermark ^^^^ equipment would require a source of real-time clock 

payload may mclude a six-bit field detailing the classes of ^ta against which the date field in the watermark can be 

devices for which the audio is authonzed. Each bit would checked to ensure that the playback rights have not yet 

correspond to a different class of device. Class 1 devices may 45 expired ) 

be personal playback devices with only analog-audio output. Another of the data fields that can be included in an audio 
Class 2 d^ces may be personal entertainment devices watermark specifies technical playback parameters. For 
1 music m digital (e.g. MP3, example, the parameter can cause the playback appUance to 
redbook, *.WAV) format, as well as analog audio. Class 3 j ^ spectral equalization that favors bass frequencies, or 
devices may be personal computer systems (i.e. with essen- so ^^^^^ frequencies, or mid-range frequencies, etc. Other pre- 
tially unlimited abihty for processing and outputting digital configured equalization arrangements can similarly be 
audio). Etc., etc. A device to which such MP3 audio is pro- ^^^^^ responsive to watermark data. Likewise, the param- 
vided would check the usage control strmg data to determme ^^er can invoke special-eflfects provided by the playback 
whether it is authorized to utilize the audio. A personal play- apphance, e.g., echo effects, reverb, etc. (Again, such param- 
back device with analog-only output, for example, would 55 ^^^^ represented in an abbreviated, coded form, 
examine the first bit of the usage control string. If it was "1 ," are interpreted in accordance with instructions stored in a 
the device would be authonzed to use (i.e. playback) the MP3 (^j^her in the playback apphance. or linked thereto), 
data; if it was a "0." the device would refiise to play the music ^^a fields and principles can be applied to non- 
In addition to pre- authorization for certam classes of ^^^^^^ ^ ^^^^^ ^ watermarked data can 
devices, the usage control string can also mc^^^ 60 adaptively control the display monitor or playback param- 
mg the number of permitted playbacks. Tins data can be ^^^^ . ^j^^ . ^^-^^ experience, 
encoded in bits seven through nine, representing eight possi- 
bilities : Music Asset Management/Commerce 

0 — no playback permitted The majority of domestic music puacy is not organized. 

1 — single playback permitted 65 Rather, it is a crime of opportunity and convenience. If the 

2 — two playbacks permitted crime were made more difficult, the ahemative of obtaining a 

3 — three playbacks permitted copy through legitimate channels would be less onerous. 
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Similarly, if the procedure for obtaining a copy through legiti- (e.g. from the clearmghouse). In either case, the library can 

mate chaimels were simplified, the incentive for piracy would provide music to the user's playback devices by short-range 

be reduced. Watermarking facilitates both — ^making the wireless broadcast. 

crime more difficult, and making legitimate music acquisition By such arrangement, a user can conveniently compile an 

easier. 5 archive of favorite music — even while away from home. 

Consider, for example, the pricing of music in conventional Many variants of the foregoing are of course possible. The 

record stores. A CD (compact disk) may cost $ 1 5, but its sale radio can be a portable unit (e.g. a boombox, a Walkman 

may be driven by just one or two popular songs on the disk. To radio, etc.), rather than an automotive unit. The UI feature 

obtain these songs, the consumers must purchase the entire employed by the user to initiate capture a musical selection 

disk, with perhaps a dozen songs of no particular interest, lo need not be a button (physical or on-screen). For example, in 

This, in essence, is a tying arrangement that benefits the some embodiments it can be a voice-recognition system that 

record labels while prejudicing the consumers. Given these responds to spoken commands, such as "capture" or "record" 

circumstances, and a ready opportunity to make copies, it is Or it can be a form of gesture interface, 

not surprising that customers sometimes make illicit copies. Instead of decoding the watermark only in response to the 

One classic technique of avoiding purchase of a complete \ s user's "capture" command, the radio can decode watermarks 
collection of music, when only one or two songs is desired, is from all received programs, and keep the most recent in a 
to record the music off the radio. While of dubious legality, smallFFO memory. By such arrangement, the user need not 
this technique was popular in the era of combined cassette/ issue the capture instruction while the song is playing, but can 
radio players. However, the desired music was sometimes do so even after the song is finished, 
difficult to encounter in a radio broadcast, and the quality was 20 In some embodiments, data corresponding to the water- 
less than superb. mark can be made available to the user in various forms . For 

The combined cassette/radio player has now evolved into a example, it can be presented to the user on an LCD screen, 

general purpose computer with wide-ranging functionality, identifying the artist and song currently playing. If a corre- 

and other sophisticated devices . Music can be a cquired 0 ff the spending UI button is activated, the device can so- identify the 

web, and can be recorded in various forms (e.g. in a personal 25 last several selections. Moreover, the data need not be pre- 

MP3 player, stored on a hard disk, stored on a writeable sented to the user in displayed form; it can be aimunciated by 

CD-ROM, played back and recorded on analog cassette, etc., known computer-speech technologies instead, 

etc.). The quality can be quite high, and the erratic broadcast In embodiments in which the watermark does not convey 

time problems of radio broadcasts have been overcome by the ASCII text data, but instead conveys UIDs, or coded abbre- 

web's on-demand delivery mechanisms. (Moreover, the 30 viations, the device must generally interpret this data before 

music can be downloaded in faster-than-realtime, a further presenting it to the user. In an illustrative embodiment, the 

benefit over recording-off-the-air techniques.) device is a pocket-sized FM radio and is equipped with a 1 

One hybrid between the new and old is a novel radio (e.g., megabyte semiconductor non-volative RAM memory. The 

for use in a car) that has a "capture" button on the &oni panel memory includes a data structure that serves as a look-up 

(or other form of user interface, e.g., a Capture icon on a 35 table, matching code numbers to artist names and song titles. 

GUI). If a user hears a song tliey want to record and keep, they When the user queries the device to learn the identify of a 

press the Capture button while the song is playing, hi song, the memory is indexed in accordance with one or more 

response, the radio device decodes a watermark embedded in fields from the decoded watermark, and the resulting textual 

the music, and thereby knows the identity of the music. The data from the memory (e.g. song title and artist) is annxmci- 

radio then makes a wireless transmission identifymg the user 40 ated or displayed to the user. 

and the desired song. A local repeater network picks up the In most applications, such memory will require frequent 

wireless signal and relays it (e.g. by wireless rebroadcast, by updating. The RF receiver provides a ready mechanism for 

modem, or other commimication medium) to a music clear- providing such updated data. In one embodiment, the radio 

inghouse. The clearinghouse charges the user a nominal fee "awakens" briefly at otherwise idle moments and times to a 

(e.g. via a pre-arranged credit card), and queues the music for 45 predetermined frequency at which updated data for the 

download to a predetermined location associated with the memory is broadcast, either in a baseband broadcast channel, 

user. or in an ancillary (e.g. SCA) channel. 

In one embodiment, the predetermined location is the In variants of the foregoing, internet deUvery of updated 

user's own computer. If a "live" IP address is known for the memory data can be substituted for wireless delivery. For 

user's computer, the music can be transferred immediately. If 50 example, the artist/song title memory in the personal player 

the user's computer is only occasionally connected to the can be updated by placing the player in a "nest" every 

internet, the music can be stored at a web site (e.g. protected evening. The nest (which may be integrated with a battery 

with a user- set password), and can be downloaded to the charger for the appliance) can have an internet coimection, 

user's computer whenever it is convenient. and can exchange data with the personal device by infrared. 

In other embodiments, the predetermined location is a 55 inductive, or other proximity-coupling technologies, or 

personal music library maintained by the user. The Ubrary can through metal contacts . Each evening, the nest can receive an 

take the form, e.g., of a hard-disk or semiconductor memory updated collection of artists/song titles, and can re- write the 

array in which the user cxistomarily stores music. This storage memory in the personal device accordingly. By such arrange- 

device is adapted to provide music data to one or more play- ment, the watermark data can always be properly intepreted 

back units employed by the user (e.g. a personal MP3 player, 60 for presentation to the user. 

a home stereo system, a car stereo system, etc.). In most The "Capture" concepts noted above can be extended to 

installations, the library is physically located at the user's other functions as well. One is akin to forwarding of email. If 

residence, but could be remotely sited, e.g. consolidated with a consumer hears a song that another friend would enjoy, the 

the music libraries of many other users at a central location. hstener can send a copy of the song to the friend. This instruc- 

The personal music library can have its own internet con- 65 tion can be issued by pressing a "Send" button, or by invoking 

nectioa Or it can be equipped with wureless capabilities, a similar function on a graphical (or voice- or gesture-respon- 

pennitting it to receive digital music from wireless broadcasts sive) user interfece. In response, the appliance so- instructed 
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can query the person as to the recipient. The person can commercialnatureofsuch websites. Instead, in an exemplary 
designate the desired recipient(s) by typing in a name, or a embodiment, the Capture feature is a software program resi- 
portion thereof sufficient to uniquely identify the recipient. dent at the user's computer. When this software program is 
Or more typically, the person can speak the recipient's name. invoked by the user, a socket channel is instantiated between 
As is conventional with hands-free vehicle cell phones, a 5 the user's computer and the clearinghouse over the then- 
voice recognition unit can listen to the spoken instructions existing internet connection. The decoded watermark data 
and identify the desired recipient. An "address book"-like and user ID is transmitted to the clearinghouse over this 
feature hasthe requisite information for the recipient (e.g., the channel, without interrupting the user's other activity (e.g. 
web site, IP address, or other data identifying ttie location to downloading music from the non-conmaercial web site). In 
which music for that recipient should stored or queued, the lo response, the clearinghouse transmits the music to the prear- 
format in which the music should be delivered, etc.) stored ranged location and attends to billing, 
therein. In response to such conunand, the appliance dis- In some embodiments, a watermark detector is included as 
patches instructions to the clearinghouse, including an autbo- part of the operating system, and constantly monitors all 
rization to debit the sender's credit card for the music charge. TCP/IP, or other internet, data received by the user's corn- 
Again, the clearinghouse attends to delivery of the music in a 1 5 puter, for the presence of watermarks. In such case, when the 
desired manner to the specified recipient. Capture feature is invoked, the program examines a memory 
Still further, a listener may query the apphance (by voice, location in which tlie operating system stores the most-re- 
GUI or physical button, textual, gesture, or other mput) to cently received watermark data. In another embodiment, the 
identify CDs on which the then-playing selection is recorded. computer does not monitor all internet traffic for embedded 
Or tiie listener may query the appliance for the then-playing 20 watermark data, but includes an API that can be called by the 
artist's concert schedule. Again, the appliance can contact a Capture program to decode a watermark from the data then 
remote database, relay the query, and forward data from the being received. The API returns the decoded watermark data 
watermark payload identifying the artist and/or song title to to the Capture program, which relays same to the clearing- 
which the query relates. The database locates the requested house, as above. In still another embodiment, the watermark 
data, and relays same back to the appliance for presentation 25 decoder forms part of the Capture program, which both 
(via a display, by machine speech, or other output) to the user. decodes the watermark and relays it to the clearinghouse 
If desired, the user can continue the dialog with a further when the Capture program is invoked by the user, 
instruction, e.g., to buy one of the CDs on which the then- ji^^^t are various techniques by which the Capture pro- 
playmg song is mcluded. Agam, this instruction may be gram can be selectively invoked. One is by a keyboard macro 
entered by voice. GUI, etc., and dispatched from the appU- 30 (g g ^ ^ combination of keyboard keys). Another is by a 
cance to the cleannghouse, which can then complete the program icon that is always presented on the screen, and can 
transaction m accordance with pre-stored information (e.g. be double-cUcked to activate. (Again, confirmation processes 
credit card account number, mailmg address, etc.). A con- called for, depending on the likelihood of inadvertent 
fuming message is relayed to the appUance for presentation to invocation.) Many other techniques are likewise possible. 

• *- • 1- 1 . . In the just-contemplated scenario, the Capture operation is 

wmie me loregoing iransacuons require a link to a remote ^^^^.^ 

user is downloading music from a non- 

nin^^SL^S^Tc 1?°^ I p """T"" coimnercialwebsite.Thisseemssome^hatredun(iant,since 

r ^ ? if - ^'^j^Pl^vl^'',^ the downloading-itself-is transferring music to the user's 

query the appliance as to the artist or song- title of the selec- ^ tt ^ ^ 

«,*Y 1 • -ru V to u^'^ y^^*^ a^iw- computer However, the Capture operation provides added 

tioncurrendyplaymg. The apphance can consult the embed- 40 ^^^^ » f f f 

ded watermark data (and optionally consult a memory to ^ \ ^ . , • 

determine the textual names associated with coded water- of streammg audio, the audio is not typically 

mark data), and providetherequestedinformation to theuser stored in a location m which it can be re-used by the con- 

(e.g., by a display, annunciation, or other output). .^^ ^ hstened-to as dehvered, but is then gone. 

me forgoing concepts (e.g. Captm^, Send, etc.) can also 45 ^he audio provides the user a copy that can be 

be employed in connection with mtemet- rather than radio- P^^V^ repeatedly. 

delivery of music. (The following discussion is illustrated ^ ^se of downloaded music files, the music may have 

with reference to the "Capture" function, but it will be rec- encoded to prevent its recordal on other devices. Thus, 

ognized that the other earlier-discussed features can be simi- ^^^^ download the music onto a desktop com- 

larly implemented.) 50 P^^^* copy-prevention mechanisms may prevent use of that 

There are many commercial web sites that sell audio (in anywhere else, e.g. on a portable music appliance. Again, 
CD form or otherwise), and offer hmited free music down- Capturing the audio provides the user a copy that can be 
loads, (or music clips) as an enticement to lure consumers. transferred to another device. (The music file provided by the 
But there are also a great number of music web sites that have clearinghouse can have copy-prevention limits of its own- 
no commercial pretense. They are hosted by music lovers 55 ^ S * the file can be copied, tut only once, or the file can be 
strictly for the enjoyment of other music lovers. When music ^P^^ only onto devices owned by the user.) 
is downloaded from such a web site, the end-user's computer (Confirmation of device ownership can be implemented in 
can analyze the digital data to decode watermark data there- various ways. One is to identify to the clearinghouse all music 
from. Again, the user can be presented with a "Capture" devices owned by a user at the time the user registers with the 
button that initiates a commercial transaction, by which a 60 clearinghouse (supplemented as necessary by later equip- 
complete copy of the then-downloaded audio is sent to a ment acquisitions). Device IDs associated with a user can be 
prearranged storage location, and the user's credit card is stored in a database ^t the clearinghouse, and these can be 
debited accordingly. This transaction can occur indepen- encoded into the downloaded music as permitted devices to 
dently of the site from which the music is downloaded (e.g. which the file can be copied, or on which it can be played.) 
through the clearinghouse referenced above). 65 The conmierce opportunity presented by non-commercial 

While the "Capture" button can be presented on the web- music web- sites is but one enabled by digital watermarks, 

site, this would generally not be in keeping with the non- There are many others. 
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To take one example, consider the media by which music 
and artists are presently promoted. In addition to radio air- 
time, these include music videos (a la MTV), fan magazines, 
web advertisements, graphical icons (e.g. the Grateful Dead 
dancing bears), posters, live events, movies, etc. Water- 5 
marked data can be used in all such media as a link in a 
commercial transaction. 

A poster, for example, typically includes a photo of the 
artist, and may comprise cover-art from a CD. The photo/art 
can be digitally watermarked witii various types of data, e.g., i o 
the artist's identify, the record label that distributes the artist's 
work, the music project being particularly promoted by the 
poster (e.g. a CD, or a concert tour), a fan web-site related to 
the artist, a web-site hosted by the record label for selling 
audio in CD or electronic form, a web-site from which free 15 
music by the artist can be downloaded, data identifying the 
poster itself, etc. 

A user, equipped with a portable apphance that merges the 
functions of pahntop computer and digital camera, can snap 
an image of the poster. The processor can decode the water- 20 
marked data, and initiate any of various links based on the 
decoded data. 

In an exemplary embodiment, after snapping the picture, 
the user invokes a software program on the device that 
exposes the various links gleaned from the snapped image 25 
data. Such a program can, for example, present the option of 
linking to the artist's fan web site, or downloading free 
streaming audio or music clips, or ordering the promoted 
CD), or requesting the above-noted clearinghouse to down- 
load a personal copy of selected song(s) by the artist to the 30 
user's personal music library, etc. (The device is presumed to 
have a wireless internet link. In devices not having this capa- 
bility, the requested actions can be queued and automatically 
executed when a link to the intemet is available.) 

Still more complex transactions can be realized with the 35 
use of a remote database indexed by digital watermark fields 
decoded from the poster. For example, the poster may pro- 
mote a concert tour. Fields of the digital watermark may 
identify the artist (by a code or full text), and a web site or IP 
address. The user appliance establishes a link to the specified 40 
site, and provides the artist identifier. In response, the site 
downloads the tour schedule for that artist, for display on the 
device. Additionally, the downloaded/displayed information 
can include a telephone number that can be used to order 
tickets or, more directly, can indicate the class of seats still 45 
available at each (or a selected) venue, and solicit a ticket 
order from the user over the device. The user can supply 
requested information (e.g. mailing address and charge card 
number) over the return channel link (wireless or wired, as the 
case may be), and the ticket(s) will be dispatched to the user. 50 
In the case of a wireless link, all of this can occiu- while the 
user is standing in front of the movie poster. 

Similar systems can be implemented based on watermark 
data encoded in any other promotional media. Consider 
music videos. Using known TV/computer appliances, water- 55 
mark data added to such videos can readily be decoded, and 
used to establish hnks to audio download, CD-sales, fan club, 
concert ticket outlet web sites, etc., as above. 

Even live events offer such watermark-based opportuni- 
ties. The analog audio fed to pubUc address or concert speak- 60 
ers can be watermarked (typically before amplification) to 
encode plural-bit digital data therein. A next generation per- 
sonal music apphance (e.g. one with a wireless interface to 
the internet) can include analog record capability (e.g. a built- 
in microphone, analog-to-digital converter, MP3 encoder, 65 
coupled to the unit's semiconductor memory). A user who 
attends a live event may record an excerpt of the music. The 
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watermark can then be decoded, and the extracted data used to 
access the Unks and commerce opportunities reviewed above. 

Cinema movies offer both audio and visual opportunities 
for watermark-based commerce opportunities. Either 
medium can be encoded to convey information of the fypes 
reviewed above. A personal apphance with image- or audio- 
capture capabihties can capture an excerpt of the audio or 
imagery, decode the watermark data therefrom, and perform 
any of the linking, etc., functions reviewed above. 

The consumer-interest watermarks reviewed above are 
only exemplary. Many others will be recognized as useful. 
For example, promotional cUps presented before a feature 
film presentation can include watermark data that point (ei- 
tiier by a literally encoded web address link, or by an ID code 
that indexes a literal hnk in a remote hnk database) to 
reviewer critiques of the previewed movies. Watermark data 
in a featured film presentation can lead to web sites with 
information about the movie stars, the director, the producer, 
and can hst other movies by each of these persons. Other 
watermark-conveyed web links can present opportunities to 
buy the movie on videotape, to purchase the movie 
soundtrack, to buy movie-related toys and games, etc. 

More on Device Control 

Much of the foregoing has focused on watermark encoding 
to provide enhanced customer experiences or opportunities. 
Naturally, watermarks data can alternatively, or additionally, 
serve the interests of the media owner. 

To illustrate, consider watermarked music. The media 
owner would be best served if the watermark sen/es dual 
purposes: permissive and restrictive. Permissively, music 
appliances can be designed to play (or record) only music that 
includes an embedded watermark signaling that such activity 
is authorized. By this arrangement, if music is obtained from 
an unauthorized soxirce and does not include the necessary 
watermark, the apphance will recognize that it does not have 
permission to use ie music, so will refuse requests to play (or 
record). 

As noted, music appliances can respond restrictively to the 
embedded watermark data to set limits on use of the music. 
Fields in the watermark can specify any or all of (or others in 
addition to) (a) the types of devices on which the music can be 
played (b) the types of devices on which the music can be 
recorded; (c) thenumber of times the music can be played; (d) 
the number of times the music can be recorded, etc. 

The device restrictions (a) and (b) can be of various types. 
In some embodiments, the restrictions can identify particular 
units (e.g. by serial number, registered owner, etc.) that are 
authorized to play/record the encoded music. Or the restric- 
tions can identify particular classes of units (e.g., battery- 
powered portable players with music memories of less than 
50 megabytes, disk-based dedicated music appliances, gen- 
eral purpose personal computers, etc.) Or the restrictions can 
identify particxilar performance quality criteria (e.g., two 
channel, 16-bit aucho at 44.1 KHz sample rate, or lower 
quahty). 

The use restrictions (c) and (d) can likewise be of various 
types. Examples include "do not copy," "copy once only," 
"unrestricted copying permitted," ^'play once," "play N 
times" (where N is a parameter specified elsewhere in the 
watermarked data, or by reference to a database indexed by a 
watermark data field), "unrestricted playing permitted," etc. 

It is straightforward to design a music appliance to respond 
to usage limits of zero (e.g. "do not copy") and infinity (e.g. 
"unrestricted copying permitted," and "unrestricted playing 
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permitted"). The device simply examines one or more bits in 
the watermark data, and permits (or refuses) an operation 
based on the value thereof. 

Implementation of the other usage-control restrictions can 
proceed in various ways. Generally speaking, the stored 5 
music can be altered to give effect to the usage-control restric- 
tions. For example, if the music is "record-once," then at the 
time of recording, the appliance can alter the music in a 
fashion indicating that it now has "do not record" status. This 
alteration can be done, e.g., by changing the watermark data lo 
embedded in the stored music (or adding watermark data), by 
changing other data stored in association with the music, etc. 
If the original signal is stored (as opposed, e.g., to a streaming 
signal, such as an internet or wireless transmssion), it too 
should be so-altered. 1 5 

Likewise with playback limitations. The number of play- 
backs remaining can, e.g., be encoded in an updated water- 
mark in the music, be tracked in a separate counter, etc. 

More particularly considering the "copy once" usage 
restriction, an illustrative embodiment provides two distinct 20 
watermark payload bits: a "copy once" bit and a "copy never" 
bit. When originally distributed (whether by internet, wire- 
less, or otherwise), the "copy once" bit is set, and the "copy 
never" bit is un-set. 

When music encoded in this fashion is provided to a com- 25 
pliant recording device, the device is authorized to make one 
copy. (A compliant device is one that recognizes encoded 
watermark data, and behaves as dictated by the watermark.) 
When this privilege is exercised, the recording device must 
alter the data to ensure that no further copying is possible. In 30 
the illustrated embodiment, this alteration is effected by the 
recording device adding a second watermark to both the 
music, with the "copy never" bit asserted. The second water- 
mark must generally be encoded in an "orthogonal" domain, 
so that it will be detectable notwithstanding the continued 35 
presence of the origiual watermark. CompUant equipment 
must then check for both watermarks, and refuse to copy if 
either is foimd to have the "copy never" bit asserted. 

One advantage to this arrangement is that if the watennark 
signal has undergone some form of cormption (e.g. scaling or 40 
resampling), the first watermark may have been weakened. In 
contrast, the second watermark will be native to the corrupted 
signal, and thus be more easily detected. (The corruption may 
also contribute to the orthogonality of one watermark relative 
to the other, since the two watermarks may not have precisely 45 
the same time base or other foundation.) 

An alternative approach is not to encode the "copy never" 
bit in the original music, but leave this bit (in whatever mani- 
festation) blank (i.e. neither "1 " nor '*0"). In transform-based 
watermark techniques, this can mean leaving transform coef- so 
ficient(s) corresponding to the "copy ne\'er" bit un-changed. 
If the watermarking is effected in the temporal sample 
domam (or spatial domain, for image data), this can mean 
leaving certain samples (pixels) unmodified. The recording 
de\ace can then alter the transform coefficients and/or 55 
samples as necessary to assert the previously-unencoded 
"copy never" bit when the permitted recording is made. 

In such a system, compliant recording devices check for 
the "copy never" bit in the sole watermark, and refuse to make 
a copy if it is asserted (ignoring the value of any "copy once" 60 
bit). 

A third approach to "copy once" is to set both the "copy 
once" and "copy never" bits, but set the former bit very 
weakly (e.g. using lower gain and/or high frequency DCT 
coefficients that do not survive certain processing). The frail 65 
"copy once" bit is designed not to siu^dve common corrup- 
tions, e.g., resampling scaling, digital to analog conversion. 
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etc. To flirther assure that the "copy once" bit is lost, the 
recording device can deliberately add a weak noise signal that 
masks this bit (e.g. by adding a noise signal in the frequency 
band whose DCT coefficient conveys the "copy once" bit). In 
contrast, the "never copy" bit is unchanged and reUably 
detectable. 

In such a system, comphant devices check for the "copy 
once" bit in the sole watermark, and refuse to make a copy if 
it is not detected as set. 

These three examples are but illustrations of many possible 
techniques for changing the rights associated with a work. 
Many other techniques are known. See, e.g., the proposals for 
watermark-based copy control systems for digital video at the 
Copy Protection Technical Working Group, http://www.dvc- 
c.com/dhsg/, from which certain of the foregoing examples 
are drawn. See also Bloom et al, "Copy Protection for DVD 
Video," IEEE Proceedings, Special Issue on Identification 
and Protection of Multimedia Information, June, 1999. 

ScaleabiUty 

One feature that is desirable in many detector embodi- 
ments is scaleability. This refers to the abihty of a detector to 
scale its computational demands to match the computational 
resources available to it. If a detector is rurming on a high 
performance Pentium HI workstation, it should be "doing 
more" than if the same detector is running on a slow micro- 
controller. One way scalability can be achieved is by process- 
ing more or less chunks of input data (e.g. temporal excerpts 
of music, or blocks/macroblocks of pixels in a frame of video 
data) to decode watermarks. For example, an input audio 
stream might be broken into chunks of one second each. A fast 
processor may complete decoding of each chunk in less than 
a second, permitting it successively to process each chunk in 
the data stream. In contrast, a slow processor may require two 
and a half seconds to decode the watermark from a chunk. 
While it is processing a first chunk, the second and third pass 
by un-decoded. The processor next grabs and processes the 
foiuth chunk, permitting the fifth and sixth to pass by un- 
encoded. 

The detector running on the fast processor is clearly more 
difficult to "fool," and yields a decoded watermark of higher 
confidence. But both systems decode the watermark, and both 
operate in 'Veal time." 

The skipping of input data in the temporal (e.g. music or 
video) or spatial (e.g. image or video) domain is but one 
example of how scaleabihty can be achieved. Many other 
approaches are known to those skilled in the art. Some of 
these alternatives rely on spending more or less time in the 
data analysis phases of watermark decoding, such as cross- 
correlation operations. 

Reference has been made to watermarked UIDs as refer- 
ring to a database from which larger data strings (e.g. web 
addresses, musician names, etc.) can be retrieved. In some 
embodiments, the data record referenced by a UID can, in 
turn, point to several other database records. By such arrange- 
ments, it is often possible to reduce the payload of the water- 
mark, since a single UID reference can lead to several differ- 
ent data records. 

Production Tools 

In the prior ait, the watermark embedded in a source mate- 
rial is typically consistent and static through a work — un- 
changing from beginning to end. But as will be recognized 
from the foregoing, there are many applications that are better 
served by changing the watermark data dynamically during 
the course of the work. According to another aspect of the 
invention, a production tool is provided that faciUtates the 
selection and embedding of dynamically-changing water- 
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mark data. One such embodiment is a software program hav- To pay for the requested video, the consumer mstructs his 
ing a user mterface that graphically displays the different or her computer to transfer three of the earher-purchased 
watermark fields that are being embedded in a work, and tokens over the web to the video provider. Various user mter- 
presents a library of data (textually or by icons) that can be face metaphors can be employed to faciUtate this transfe^ 
inserted into each field, and/or permits the user to type in data 5 e.g., pemiitting the user to type the amount of money to be 
to be encoded. Another control on the UI controls the advance transferred in a dialog box presented on-screen, or droppmg/ 
and rewind of the media, permitting the user to determine the dragging icons representing tokens from an on-screen wal- 
location at which different watermark data begins and ends. let" to an on-screen 'ticket booth" (or over an icon or thumb- 
Graphical paradigms known from video- and audio-editing nail representing the desu-ed content), chckmg on an 
tools can be used to indicate the starting and ending frames/ lo "increment" counter displayed adjacent the Usting of the con- 
samples for each different watermark payload. tent, etc. Once the consumer has authonzed a transfer of 
Such a tool can be of the standalone variety, or can be sufficient tokens, the consumer's computer sends to the web 
integrated into the desktop audio- and video- production and site (or to such other web address as HTML encoding m the 
editing tools offered by vendors such as Avid, Adobe, Jaleo, viewed web page may indicate) the tokens. Tins transmission 
Pinnacle Systems, SoundForge, Sonic Foundiy, Xing Tech- is simply takes the form of the three 128+ bit numbers (the + 
nology, Prosoniq, and Sonic Desktop Software. indicating the bank identifier)--in whatever packet or otiier 

format may be used by the internet Imk. Once dispatched m 
Payment-Based Systems manner, the tokens are deleted from the user' s computer. 

Another aspect of the present invention is the use of anony- qj. simply marked as spent. (Of course, in other embodiments, 
mous payment tokens that can be used to obtain content on the 20 a record of the expenditure may be stored in the consumer's 
web. In one embodiment, a token comprises a 1 28-bit pseudo- computer, e.g., with the token contents and a record of the 
random number, to which additional bits identifying an issu- ^l^^i^q video purchase to which they were applied.) 
ing bank (or other issuing institution) are appended. (The Smcc the amount of money is nominal, no encryption is 
additional bits can be the IP address of a web server of the provided in this embodiment, although encryption can natu- 
bank, a routmg number identifying the bank for electronic 25 rally be provided in other embodiments (e.g., either m send- 
wire transfers, or other identifier.) The 128-bit numbers are i^g tokens from the user to the web site, or earlier, in 
randomly generated by the bank— commonly as needed— sending the tokens to the user). As will be seen, provided that 
and eachrepresents a fixed increment of money, e.g. ten cents. tj^g media provider immediately sends the tokens to the bank 

A consumer wishing to have a store of currency for such in real time, encryption is a nice feature but not mandatory 
commerce pays the bank, e.g., $10 in exchange for 100 30 On receipt of the token data, the web site immediately 
tokens. These tokens are transferred electronically to disk or routes the token data to the identified bank, together with an 
other storage in the consumer's computer in response, e.g., to identifier of the media provider or account to which the funds 
a credit card authorization, or may be provided by diskette or represented thereby are to be credited. The bank checks 
other storage medium over the counter at a bank branch (in whether the 128-bit numbers have been issued by that bank, 
which case the consumer thereafter copies the numbers into 35 and whether they have afready been spent. If the numbers are 
storage of his or her computer). (Outlets other than banks can valid, the bank updates its disk-based records to indicate that 
of course be employed for distributmg suchnumbers, much in the three tokens have been spent and that the bank now owes 
the manner that convenience and many grocery stores com- the media supplier 30 cents, which it may either pay iname- 
monly issue money orders.) diately (e.g., by crediting to an account identified by the 

Imagine that the consumer wishes to view the final quarter 40 media provider) or as one lump sum at the end of the month, 
of a Trailblazer basketball game that aired on television a The bank then sends a message to the web site confirming that 
week ago. (The consumer may have either missed the game, the tokens were valid and credited to the requested accoimt. 
or may have seen it but wants to see the last quarter again.) (Optionally, a message can be sent to the purchaser of the 
The user directs a web browser to a web site maintained for tokens (if known), reporting that the tokens have been 
such purpose and performs a search to identify the desired 45 redeemed.) 

program. (Typically, the web site is maintained by the propri- In response, the web site begins deUvery of the requested 
etor that holds the copyright in the material, but this need not video to the consumer. In the illustrated embodiment, the 
be the case. Some material may be available at several web video is watermarked prior to delivery, but otherwise sent in 
sites, e.g., maintained by ABC Sports, the National Basket- unencrypted fashion, typically in streaming format, but 
ball Association, and Sports niustrated.) The search can use 50 optionally in file format. (Encryption can be used in other 
any of various known search enguies, e.g.. hifoseek. Verity, embodiments.) The watermarking in the illustrated embodi- 
etc, and can permit searching by title terms, keywords, date ment is accomplished on-the-fly and can include various data, 
ofairing,copyrightowner,etc.Bytypingin,e.g.,thekeyword includmg the date of downloading, the download site, the 
'Trailblazers' and the date 'Apr. 26, 1999,' the consumer is destination IP address, the identity of the purchaser (if 
presented a listing of videos available for download. One, 55 known), etc. 

hopefully, is the requested game. With each listing is an The large size of the video and the small charge assessed 
indication of an associated nominal charge (e.g. 80 cents). therefor provide disincentives for the consumer making illicit 
On clicking on a hypertext Unk associated with the desired copies, (Especially as to archival material whose value decays 
basketbaU game, the viewer is presented a fiirther screen with with time, there is not much after-maricet demand that could 
one or more options. The first of the Usted options is the entire 60 be served by illicit copies, making third party compilation of 
game, with commercials. The charge is the nominal charge such material for re-distribution financially unattractive. Fu^t 
presented on the earUer screen (i.e. 80 cents). Other options run video, and material that keeps a high value ovct time 
may include the first, second, third, and fourth quarters of the would not be as well suited for such distribution, and could 
game individuaUy, each of which— save the last, costs 20 better employ technology disclosed elsewhere lierem.) 
cents Tlie last may be charged at a premium rate, e.g., 30 65 In some embodunents, the integrity of the received video is 
cQits' CUcking on the desired video option yields a fiirther checked on receipt. TTiis feature is descnbed below m the 
screen through which payment is effected, section entitled Watermark-Based Receipts. 


us 7,562,392 Bl 

23 24 

In the iUustrative system, nothing in the tokens indicates corresponding field. If the search reaches the end of the linked 

the identity of the purchaser. The web site knows the IP chaia without finding a match between the stored numbers 

address of the site to which video was delivered, but need not and the token number, the token is treated as mvaUd (i.e. not 

otherwise know the identity ofthe purchaser. The bank would issued by that bank). 

probably maintain a record of who purcliased the tokens, but 5 Other manners of tracking the large number of possible 

need not. In any event, such tokens could thereafter be token numbers can of course be used; the foregomg is just 

exchanged among consumers, resulting in anonymity from exemplary. Or the tokens needn't be tracked at all. Such an 

the bank, if desired. arrangement is highly practical if the token has sufficient bits . 

As described above, the video excerpts from which the With the illustrated 128 bits, for example, the chance of two 

consumer can select include commercials. At some sites, lo identical tokens being issued is infinitesimally small, so 

video may be provided from which the commercials have checking for duplicate issuance can be omitted if desired, hi 

been excised, or which is delivered in a manner that skips past such case, the bank can simply mamtam an ordered hst ofthe 

the commercials without transmitting same to the consumer token numbers still outstanding and valid. As new tokens are 

Such video will naturally command a premium price, hi some dispensed, their token numbers are added to the list. As tokens 

embodiments, the difference in price is electronically cred- 1 5 are redeemed, their numbers are deleted from the Ust. Known 

ited as compensation to accounts mamtained for (or by) the list processing techniques can be employed to speed such 

advertisers, whose advertisements are not being viewed by search, update, and delete actions, 

such consmner^. (The identification of advertisers to be cred* w^teimark-Based Receipts 

ited IS desirably permanentty video, either p ^..^^.tent applications commonly assume that if 

throughout the video (if ttie video has had the commercials 20 is transmitted from a server (or h^d-end, etc.), it is 

removed herefrom), or by data m the commercials them- ^^^^^^ ^^^^^^^^^ assmnption is wrong, 

selves (which coimnercids are slapped for transmission to ^ interruptions and internet traffic load 

tiieconsumer,butcanstillbedecode^ dLnish (e.g., dropped ^deo frames), or even negate 

Such encodmg can be by m-band watermarkmg or other- ^^^ ^ failed deliveiy), exacted consumer enjoyment of con- 

r • ^11 r^^.'A^^^A tent, hi such cases, the consumer is left to haggle with the 

While tiie foregomg discussion particularly considered ..^^ent provider in order to obtain an adjustment, or refimd, 

video as the desned content, the same prmciples are equally f Tl' rhflTxrec 

applicable in connection with audio, stm imagery, and other ^™~^^deamechanismforconfinningreceiptof 

'°Soken-based payment method is but one of many that 30 content If a watermark is ^ftected <^;i^"^^^^ 

can be employed tfelitera^relating to 0.^^^^^^^^ ^raJe'rceT^nl ^^^^^^^ 

mechamsms IS exiensive. and aU such systems can genemUy ^^^^^.^t^.^^^pg^iy delivered. THis receipt can be stored. 

Sk^lSttokenscanbealogisticalproblemforthe and/or sent to the content distributor to confirm dehv^^^^ 
bank.Oneapproachistohaveamemorywi1iilO'-locations. 35 In one mbodmient, a content rec«vmg d^ce (e g com- 
and at each location store a two-bit value (e.g. 00=never puter television or set-top box audio apphan^^^^^ etc.) pen- 
issued;- 01 =issued but not spent; 10=issued and spent; odi^Uy decodes a watermaric ^^^'^'"^^^^^1'° 
ll=i«served). More complete data could alternatively be 

stored.butsuchamemory wouldbeimpracticallylarge. onds a ^^^^ de^^^^or c^ d^^^^^^^ 

One alternative approach is to hash each 128-bit number. 40 a record of the decoded data (or s^P^V '^""^d &e fe^ 
when issued, to a liuch smaUer key value (e.g. 20 bits). A of continued detection of flie ^J^^ J^™^)^ ^en a 
memorywithlO^locationscanbeindexedbythiskey.Each '^l^S^'l ? ^ ^ Tth^^^^^^ 
such location can include four data: an issued 128-bit token « object begms) the duration of the previously-re- 
number that hashes to that value, first and second date fields reived content is logged, and a receipt is issued, 
indicating the date/time on which that token was issued and 45 In a related embodiment the last portion (e.g.. 3 seconds, 
redeemed, respectively, and a link specifying the address of a fi^me, etc.) ofthe content bears a dififer^t end of content 
nextmemory locationTliatnextmemoiy location(outsideof watermark that triggers issuance of a receipt Such a water- 
the original 10^ locations) can include four more data, this mark can mdicate the length of the contait, to ser^^ as a 
time fora second issued-128-bit token number that hashed to cross-check agamst the penodic watermark pollmg. (Jj. if 
theoriginalkeyvalue,twodatefields,andagainwithalinkto so P^^^ic smphng at 2 second lnter^^ls yiel^^ 
p .,ih<:Muent <aoraBe location etc correspondmg to the same content, and if the end of content 
" aT28.5LSn5ntr is generated, the original watermark indicates that the content was 1^32se«)nds long, 
memory location indexed by the hash code of that nmnber is then receipt ofthe entu-e content can be confirmed.) 
checked for an earlier number ofthe identical value (to avoid In another embodunent, the watermark can change during 
issuance ofduplicate tokens). Each successive location in the 55 the course ofthe content by mcludmg, e.g., a datum that 
Unked diain of memory locations is checked for the same increments every fi:ame or other mcrement ot time (e.g., 
1 28-bit number. When the end ofthe linked chain is reached, frame number, time stamp, etc.). A watermark detector can 
the bank knows that the 128-bit random number has not monitorthecontinuedincrementmgofthisdahimtooughout 
previously been issued, and- writes that number in the last- the content to confirm that no part was garbled (which would 
addressed location, together with the date of issuance, and a eo destroy the watermark) or was otherwise missmg. Agam, at 
Unk to a next storage tocation. the end of delivery, the receiving system issue a ojnfir- 
When a 1 28-bit token is received, the same Unked-list mation that XXX fi-ames/seconds/etc. of the identified con- 
processing occun to identify a first location, and to thereafter tent were received. 

step through each subsequent location until a match is found One appUcation of such technology is to bill for content 

between the token number and the number stored in one of the 65 based on receipt, rather than transmission. Moreover, biUings 

Unked memory locations. When found, that number is can be adjusted based on percentage of content-value 

marked as redeaned by writing a redemption date/time in the received. If delivery is interrupted mid-way through (e.g., by 
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the consumer disabling the content-receiving device), the don't originate through the same server, time/frame data can 

nominal billmg for the content can be halved. Some pro- be relayed as necessary to the alternative soundtrack server to 

longed content, e.g., televised/web-broadcast university effect synchronization.) 

classes, cannot be "consumed" in one session, and are thus Another application of the Master Global Address is to 
particularly well suited for such pay-as-you-consume billing. ^ serve as a point to which monitoring stations can report the 

Another application of such technology is in advertising P^^'^^f ^f content. Consider, for example, a 

verification Presentlv ads are tracked bvtan<imission or less copynght-aware node through which content signals pass, 

venncauon_ msenuy.aos aretrackedby transnnssiono^^^ ^ computer node on a network, a satelUte transponder, 

frec^uently byd^ectionofanemteddedcod^^^^^^^^ etc. When^er the node detects passage of a media object 

Niehen Media Research s Pat. Nos. 5,850.249 and 5,737, (e.g.. by reference to a file extension, such as MPS, JPG, AVI, 

025). However, such reception-detectors— once triggered- g^^. it sends a "ping" over the internet to the address encoded 

generaUy do not fiirther note the length of tmie that the adver- uj object, simply reporting passage of the object. Similar 

tismg was received, so the same data is produced regardless of monitoring facUities can be provided in end user computers, 

whether only five or fifty seconds of a commercial is pre- e.g., reporting FileOpen, FileSave, Printing, or other use of 
sented.Watermaik monitoring as contanplated herein allows 15 content bearing MGA data. 

the duration of the advertising impression to be precisely This system can be expanded to include "ping" and "pong" 

fracked. phases of operation. When a software application (or a user 

In one apphcation of this technology, recipients of adver- appUance, such as a video or audio playback device) encoun- 

tising are provided incentives for viewing advertising in its '«rs a media object (e.g., at time of file open, at time of 
entirety. For example, a content-receiving device can include 20 playback, etc.), it pings the MGA site to report the encounter, 

a watennaric detector that issues a receipt for each advertise- MGA site "pongs" back, responding with mstructions 

mentthatisheard/viewedinitsentirety.niesereceiptscanbe appropriate to the encounter. For example if the object 

redeemed, e.g.. for content tokens as described elsewhere ^qimes payment of a fee before fimfimctionahty or a 

herein, for monetary value, etc. hi some embodiments. '°Jf -^f^J site can respond to the apphcation 

receipts are generic and can aU be applied to a desired pie- mstructions that the object be used (e.g., played back) 

mium, regardless of the advertisements through which ihey ""^^ ^"^^ ^"PPj'^ state preventmg the user s fiJl enjoy- 

were earned. In other embodiments, thereceipts areassoci- ment(e.g., impaired j^olution, or mjpau-edsom^ 

ated with the particular advertisers or class of advertisers). ^"^.'P^^ ^-^y- f ' " '''^ "^..f" f Tfi^fi"""' 

THUS, a TV viewer who accumulates 50 receipts from adver- fP^f^°^ °u*lf ™!,^'t^- ^T^?*^ ^ 
tismg originating from Procter & Gamble may be able to '° tonality can be obtamed. The apphcation can ^aphically 

redeem same for a coupon good for $2.50 off any Procter & ''^^^^^ P^«^«°^ ""^^'1^. ° user who can autho- 

Gamble product, or rLipts from Delta AirUnes may be a payment, ifdesired, so that toe content can be enjoy«^ 

redeemedforDeltafrequencyfliermiles(e.g.,atarateofone a less- (or un-)c^ppled state. On receipt of the p 

mile per minute of advertising) Such incenti es are arti authorization, the MGA site can mfonn the user apphcation 
larly usefiil in new forms of media that give the consular " M f /^^^.'^ access/usage rights have been purchased, and 

enhanced opportunities to fast-forwanl or otherwise skip that the apphcaUon may proceed accordingly^ 

advertising Yet another apphcation of the MGA is to present the user of 

,,,^ \,r> . , . a content object a menu of options that is customized to that 

(Although the foregomg "receipt" concept has been object 
described in conjunction with watermark data (and use of 40 in current graphical operating systems, when a user clicks 

watermark technology is beheved to be inherently advanta- ^ ^he right mouse button), a menu is 

geous m this application), the same pnnciples can hkewise be presented detailing actions that can be undertaken in connec- 

implemented with ancillary data conveyed by other means.) the icon, or the file represented thereby. Such 

Master Global Address options are pre-programmed (i.e., static), and are typically 
. ^ J V V J • ui . t. • r. 45 determined by the operating system based solely on the file 

As suggested above, it is desirable that each piece of con- extension ^ ^ 

'Zr^^^ '"^^^ ad^ess (the 'faster Global Address" accordance with this aspect of the present invention, 

(MGA), or Master IP Address associa^^^^ Such cUcking on an icon representing a media ^ct initiates an 

address is typically conveyed with the content, e.g., by an IP i^temeFlink to the MG A site associated with the object. Tlie 

aaaress watennarKea merem. j^^^ ^.^^ responds with data that is used to customize the 

Consider a consumer who downloads a streaming video menu of options presented to the user in connection with that 

having an English language soundtrack. The viewer may not particular object. 

speak Enghsh, or may otherwise prefer to Usten to the Consider an icon representing a JPG image file. Right- 
soundtrack in another language. The user can decode the clicking on the icon may yield a menu that gives the user 
watermark data embedded in the video and initiate a hnk to 55 various options presented by the operating system (e.g., 
the associated web address. Therethe user is presented with a delete, compress, rename), and additional options custom- 
Ust of soundtracks for that content object in other languages. ize^ in accordance with data from the object's MGA site. 
The viewer can click on the desired language and receive These customized options may include, e.g., 
same via a second simultaneous h^smission (e.g., a second (a) open in 1 00x1 50 pixel format for fi-ee; 
socket channel). The consumer's audio/video appUance can (b) open in 480x640 pixel format for ten cents; 
substitutethe desired audio track for the default English track. (c) open in 960x1280 pixel format for twenty cents; 

If the streaming video and the alternative soundtrack are (d) purchase rights to use this image in a newsletter having 

hosted on the same server, synchronization is straightfor- a circulation of under 1000 for $1.25; 

ward. The process governing transmission of the alternative (e) display a complete listing of license options, 
soundtrack identifies the process that is streaming video to the 65 Clicking on options (b) or (c) initiates a commerce apph- 

same IP address. Based on SMPTE, or other time/ftame data, cation through which fUnds are electronically transferred to 

the former process syncs to the latter. (If the two data streams the MGA site (by the above-described tokens or otherwise). 
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In response, the MGA site responds (e.g., with TCP/IP or 
HTML instructions) authorizing an application on the user's 
computer to open the file in the requested manner. (The 
default application for JPG appUcations can then automati- 
cally be launched, or the computer may first query the user 5 
whether another application should be used instead.) 

' Clicking on option (d) proceeds as above, and permits full 
use of the image on the computer. Moreover, the MGA site 
sends a digital certificate to the user's computer memorializ- 
ing the usage rights purchased by the consumer. i o 

La this particular arrangement, no access control is placed 
on the content, e.g., by encryption, secure container technol- 
ogy, or the like. The nominal fees, and the ease of Ucensing, 
make it simple for the user to "do the right thmg" and avoid 
copyright liability. In other embodiments, of course, known is 
access control techniques can be used to limit use of the obj ect 
until the requisite payment has been made. 

Naturally, records of all such transactions are also logged at 
the MGA site. 

Clicking on option (e) opens a browser window on the 20 
user's computer to a web site that presents a complete hsting 
of license options available for that image. (The address of 
this web site is included in customization data relayed to the 
user device from the MGA site, but not explicitly shown to the . 
user on the menu.) Through such web site, the user can select 25 
desired rights, effect payment, and receive the necessary 
authorization for software apphcations on the user's com- 
puter (or other media appliance) to open and/or process the 
content. 

The object on which the user "clicks" needn't be an icon. It 30 
can be an image or other graphical representation. (And a 
"click" isn't necessary; a voice command or other signal may 
be xised to the same effect with an audio chp or selection.) 

Consider the popular merchandising of books and CDs 
over the internet. A JPG or other image file depicting the cover 35 
of a book, or the artwork of a CD cover, can be treated as a 
media object, and can include a watermarked MGA pointer. 
Right-clicking on such an image of a book cover could, 
through the MGA site, present to the user a menu of options 
that includes — in addition to those normally presented in 40 
conjunction with a JPG file — the following: 

(a) "See the review of this book pubhshed in the New York 
Times on Apr. 19, 1999" 

(b) "See the Ust of reviews of this book at Ainazon.com" 

(c) "Enter your own revdew of this book, for posting on 45 
Amazon.com" 

(d) "See today's sales rank of this book atAmazon.com" 

(e) "Purchase this book from Amazon.com for $ 16.95" 
(0 "Purchase this book from Bamesandnoble.com for 

$19.95 and receive a $5.00 credit towards your next so 
purchase" 

(g) ''Link to the web site that tells about the release of this 
title as a motion picture (presently scheduled to open on 
Oct. 10, 1999)" 

(h) '*Link to the Yahoo listing of web sites relating to this 55 
book" 

(i) "Search Lycos for Ustings relating to this book." 

If the user selects one of the purchase options from the 
menu, a pre-stored e- commerce profile - containing the user 
name, credit card nxmiber, billing address, ship- to address. 60 
etc., possibly in the form of an encrypted object could be 
sent to the MGA site (or to the bookseller) to effect the 
purchase, or such selection could initiate display of additional 
screens or sub-menus through which the user would manually 
enter or select such information for transmission. 65 

Others of the selections cause a new browser window to 
open on the user's computer, opening to a URL specified in 
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data relayed fix)m the MGA site but not displayed to the user 
in the menu. Appropriate HTML instructions can be gener- 
ated to effect a particular query or other operation at the 
specified URL. 

In some embodiments, the customized menu presents only 
a single choice in addition to those normally provided by the 
operating system, e.g., "Link to home." Clicking on this 
option opens a browser window to a home page at the MGA 
for that object. On that page, the user is presented with all of 
the foregoing options, and more (possibly including advertis- 
ing graphics or multi-media). Such objects can serve as pow- 
erfiil marketing agents. Returning to the example discussed 
above, a JPG image file of a book cover may have, as its 
MGA, a web page hosted by a particular bookseller, provid- 
ing purchase options and other information for that book. 
Marketing of books (or CDs, or cars, or consumer appliances, 
or virtually anything else) can be effected by disseminating 
such vendor- issued IPGs as widely as possible. Some book 
cover JPGs may be distributed by Amazon.com, otiiers by 
Bames&Noble.com, others by Borders.com — each pointing 
back to a different MGA through which purchase transactions 
for that book may be performed. 

Returning to the MGA-customized menus, these needn't 
be limited to menus resulting from clicking on an icon or 
image (or signaling during an audio excerpt). Drop-down 
menus in appHcation programs can likewise be populated 
with customized options, in accordance with customization 
data obtained from the MGA site for the object presently 
being accessed or used. Most graphical operatmg systems and 
apphcation programs have well developed toolsets permitting 
such menu customization. Again, other data relayed from the 
MGA site is not shown to the user, but is employed by the 
computer (e.g., a browser program) to carry out menu options 
selected by the user. 

Again the foregoing techniques are equally applicable for 
still images, audio, video, and other forms of content, and can 
readily be adapted for use both with general purpose comput- 
ers, software applications, and specialized media appliances. 

While, for expository convenience, the foregoing discus- 
sion contemplated embedding a literal URL address in the 
object as the MGA, more typically this is not the case. 
Instead, the MGA more commonly comprises identification 
data for the object (e.g. a 128-bit random ID), together with 
the URL for a name server computer that serves many (per- 
haps millions) of such objects (an example of the latter is the 
Digimarc MarcCentre server). 

To obtain the desired data as detailed above, tiie user's 
computer (sometimes termed a client computer) links to the 
name server computer and provides the ID of the object being 
processed. The name server computer uses this ID to query.a 
database, and obtains from the database the current IP address 
to which such queries should be routed. The name server 
computer can relay the request from the chent computer to the 
correct destination address, or can return the correct destina- 
tion address to the client computer, which can initiate such a 
link itself By such arrangement, the IP address \iltimately 
associated with an object can be easily changed as needed, 
simply by changing the corresponding record in the name 
server database, without rendering obsolete legacy objects 
having out-of-date addresses encoded therein. 

In some embodiments, the URL of the name server needn't 
be included in the watermark. In the absence of a specified 
URL, the client computer may direct such hnks to a defeult 
name server address instead (stored locally or remotely). If 
that server doesn't recognize the object ID, it can return an 
error code, or pass the query on to other name servers. Those 
servers, in turn, can pass the query along to still other name 
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servers if tliey don't recognize the object ID. In this fashion, pirates. In one embodiment, a computer's operating system 

an exponentially- large number of name servers might be (including peripheral device drivers) monitors various data 

quickly polled for information relating to the identified within the system (e.g., data sent to writeable storage media, 

object. Alternatively, rather than encoding the complete IP or sent via a serial port or network connection, etc.) for data 

address of the name server in an object watermark, the first N 5 bearing a do-not-copy watermark. The presence of such data 

(e.g., 1 6) bits of the object ID might be used as a short-hand being sent, e.g., to a writeable disk or to a remote computer, 

for one of 65,536 predetermined name server addresses, in indicates that the do-not-copy instruction has been circum- 

accordance with data stored locally (e.g., on RAM or disk in vented. In such case, the operating system writes one or more 

the user's computer) or remotely (e.g., at a default name covert records memorializiag the activity, for possible use in 

sender IP address). lo criminal prosecution if the computer is lawfiiUy seized. 

While the basic concept idea behind embedding MGA data The example just-provided is but one of many monitoring 

within an object is to point to a repository of data about the and response techniques that may be employed to deter cir- 

object, a pointer the other way may be achieved as well. cumvention of copy-protection or other access control sys- 

As noted, the "pmg" application of MGA data pennits an terns. Generally spealdng, if content data is found where it 

MGA site to be mformed of sites through which its object 15 shouldn't be, or is found used as it shouldn't be used, a 

passes. More generally, the MGA site can log the originatmg corresponding record should be made. (Other intervention 

address of each query it receives. Each such address can be actions can be triggered as well; covert tracing is desirably 

presumed to have (or have had) a copy of the corresponding just one of several parallel responses to suspected hacking.) 

object. Media owners can thereby track the dissemination of Meta-Data Accessed Using Watermarks 

copies of men- media objects-at least msofar as use of such 20 j^^ta-data, in formats knownas XML, SGML, and HTML, 

objectsentailscommumcatmgwiththeassociatedMGAsite. ^^^^^^ communicate information about digital 

Suchtrackmgoffersagreatni^^^^ some objects (e.g., author, keywords, price, rights, caption, etc.). 

m the area of comma-ce. The MGA site coTxespondrng to the j^Q^e generally, meta-data can be thought of as any data 

cover art of a Garth Brooks CD, for example, can provide a ^^^^ associates the name of a property (e.g., 

hstmg of IP addresses of persons mterested m tiiat CD. EmaU 25 u^^^^^^^ ^^^^ ^he value of the property (e.g., "Mark Twain"), 

or promouonal data objects (e.g., audio clips) can be sent to guch data commonly appears in a tag fonnat, such as the 

that hst of addresses when a subsequent Garth Brooks CD is followin^^ • 

released. ^ 

Such tracking also opens up a new dimension of internet <MEmNME-''author"coNTENT='-Mw-k 
searcliing. Presently, tntemet search engines use a brute force 

approach, visitingmiUionsofpagesacrossthewebinorderto Meta-data is commonly exchanged between server and 
identify, for example, a dozen instances ofa given photograph client computers in conjunction with the digital objects to 
file. MGAs offer a shortcut to such brute force approaches. which they relate (e.g., the text ofa Mark Twain book). 
With the present technology, a search engine can find a single As detailed herein, an important application of watermark- 
instance ofa photograph file and, by detection of the MGA 35 ing is likewise to convey information about media— in this 
data watermarked therein, link to tiie corresponding MGA case embedded within the media content itself (e.g., provid- 
site. From the MGA site, the search engine can obtain a listing ing unique identification, establishing some basic behaviors 
(if such queries are authorized) of some or all of the other sites such as do not copy, and providing Unks to extended func- 
known by the MGA site to have copies of that photograph file. tionahty). 

(Providing such data to search engines is a commerce oppor- 40 For meta-data to be useful, it must be hnked to associated 

tunity for such MGA sites, which may permit such access to content, whether in the context of a browser, appUcation 

its listing of sites only in exchange for a fee. Or the MGA site program, operating system, asset management system, search 

may arrange to collect a tribute payment fi-om the search engine, etc. However, as detailed below, tiie content and the 

engine proprietor each time the engine responds to a user associated meta-tags needn't always be conveyed together 

query using data collected fix)m the MGA site.) 45 Consider an appUcation program or other client process 

Many of the addresses logged by the MGA may not be that receives a watermarked media object. The watermark 

publicly-accessible data stores. The search engine can check includes an MGA for that object (which, as noted above, may 

each listed address to ensure that the desired object is present ^ot specify an ultimate IP address). Stored at the MGA site is 

and accessible before adding the address to its database. meta-data corresponding to the object. By linking to the 

. 50 MGA site identified by the objects watermark, the cUent 

Covert Tracmg computer can obtain the meta-data corresponding to the 

Co-pending appUcation Ser. No. 09/185,380 describes object. This data can be stored at the client computer and used 

anti-counterfeiting technology that looks for the presence of just as any other meta-data, e.g., to define the local fimctions 

digital data corresponding to bank note imagery in a com- that should be available for use with that object (e.g., buy, 

puter system, and makes a covert record of any attempt to 55 search, etc.) 

process such data (e.g., Scan, FileOpen, FileSave, Print, Edit, A particular example is an on-line catalog of stock photog- 
etc). Such records are hidden from the user of the system raphy. Each photograph is watermarked with MGA data. To 
(using, e.g., various data encryption and obscuring tech- identify tiie photographer, copyright date, price, telephone 
niques), but authorized law enforcement officials are pro- number, subject, etc., an appUcation program can link to the 
vided tools by which these records can be recovered. The eo MGA site for that photograph, and obtain the corresponding 
forensic data thereby obtained may prove usefiil in prosecut- meta-data. This data can then be displayed or used as needed. 
iQg counterfeiters. (Knowledge that a computer may be Data objects ofdisparate formats thus can readily be handled 
covertiy storing evidence of attempted counterfeiting actions within a single, simple appUcation program, since the pro- 
may prove as. or more, x^aluable in detoring counterfeiting gram needn't concem itself with the varying formats for the 
than the covert records themselves.) 65 associated meta-data (assuming the name servers provide this 
The same techniques can be employed to deter unautho- data in standardized format). Substantial flexibility in pro- 
rized processing of audio, image, video, or content by media gramming and object formatting is thereby achieved. 
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Returning to the internet search engine example described 
above, MGAs may become recognized as repositories rich m 
meta-data for media objects. Specialized search engines may 
focus their data collection around such sites, and be able to 
quickly identify the MGA sites corresponding to various 
boolean combinations of meta-tag parameters. 

Asset Management/Containers 

Much has been written on the topic of asset rights manage- 
ment. Sample patent documents include U.S. Pat. Nos. 5,892, 
900, 5,715,403, 5,638,443, 5,634,012, 5,629,980 and laid- 
open European application EP 862,318. Much of the 
teclinical work is memorialized in journal articles, which can 
be identified by searching for relevant company names and 
trademarks such as IBM's Cryptolope system, Portland Soft- 
ware's ZipLock system, the Rights Exchange service by Soft- 
bank Net Solutions, and the DigiBox system from InterTrust 
Technologies. 

An exemplary asset management system makes content 
available (e.g. from a web server, or on a new computer's hard 
disk) in encrypted form. Associated with the encrypted con- 
tent is data identifying the content (e.g. a preview) and data 
specifying various rights associated with the content. If a user 
wants to make fiiller use of the content, the user provides a 
charge authorization (e.g. a credit card) to the distributor, who 
then provides a decryption key, allowing access to the con- 
tent. (Such systems are often realized using object-based 
technology. In such systems, the content is commonly said to 
be distributed in a "secure container.") 

Desirably, the content should be marked (personalized/ 
serialized) so that any ilUcit use of the content (after decryp- 
tion) can be tracked. This marking can be performed with 
watermarking, which assures that the mark travels with the 
content wherever - and in whatever form -- it may go. The 
watermarking can be effected by the distributor prior to dis- 
semination of the encrypted object— such as by encoding a 
UID that is associated in a database with that particular con- 
tainer. When access rights are granted to that container, the 
database record can be updated to reflect the purchaser, the 
purchase date, the rights granted, etc. An alternative is to 
include a watermark encoder in the software tool used to 
access (e.g. decrypt) the content. Such an encoder can embed 
watermark data in the content as it is released from the secure 
container, before it is provided to the user. The embedded data 
can include a UID. This UID can be assigned by the distribu- 
tor prior to disseminating the containo-. Alternatively, the 
UID can be a data string not known or created until access 
rights have been granted. In addition to the UID, the water- 
mark can include other data not known to the distributor, e.g. 
information specific to the time(s) and manner(s) of accessing 
the content. 

As noted earUer, access rights systems can be realized with 
watermarks without containers etc. For example, in a trusting 
world, copyrighted works can be freely available on the web. 
If a user wishes to make lawful use of the work, the user can 
decode its watermark to determine the work's terms and 
conditions of use. This may entail linking to a web site speci- 
fied by the embedded watermark (directly, or through an 
intermediate database), which specifies the desired informa- 
tion. The user can then arrange the necessary payment, and 
use the item knowing that the necessary rights have been 
secured. 

Remote Reconfiguration of Watermark Detectors 

la some cases, it is desirable to reconfigure watermark 
detectors remotely. Such fimctionality is desirable, for 
example, if a watermark system is hacked or otherwise com- 
promised. 
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In accordance with this aspect of the present invention, 
some aspect of a watermark detector's operation is changed in 
response to a command. The change can take various forms. 
In watermark systems employing pseudo-random key data 

5 (e.g., spread spectrum spreading signals), the pseudo-random 
signal used for detection can be changed. In systems using 
DFT processing, the mapping between message bits andDFT 
coeflicients can be changed. In still other systems, the decod- 
ing can proceed as before, but the significance of one or more 

10 bits can be changed (e.g., bits that were normally interpreted 
as defining Field A can be interpreted as defining Field B, and 
vice versa). In yet other systems, the decoding can proceed as 
before, but the response of a device to a given watermark 
signal can be changed. In still other systems, a set of software 

1 5 instmctions can be re- written or re-ordered to effect a change 
in detector operation. 

The command can be conveyed in various ways. In one 
embodiment, it can be a trigger bit in the watermark payload. 
Normally the bit has a value of "0 ." If the bit has a value of" 1 

20 the detector system responds by changing its operation. A 
trigger pattern can also be established, so that detection of a 
certain combination of bits in the watermark payload serves 
to trigger the change. Reserved states of certain data fields are 
examples of patterns that might be employed. 

25 The command can also be conveyed through another chan- 
nel different than the watermark charmel (e.g., an SCA chan- 
nel of an FM broadcast, or the sub-titling data channel of 
video broadcasts, or header data within an MPEG data 
stream, etc., etc.). 

30 The change can proceed in accordance with a pre-pro- 
grammed rule (e.g., codes progressing successively through a 
numerically or algorithmically-determmed progression), or 
the change can proceed in accordance with data specified 
elsewhere in the payload of the watermark bearing the trigger 

^5 bit (e.g., instead of being interpreted in normal fashion, the 
non- trigger bits of the detected watermark can define a new 
pseudo-random key data. Or the change can proceed in accor- 
dance with data conveyed in successively-presented water- 
mark payloads, as might be done m video encoding where 
each frame of video can convey fiirther watermark informa- 
tion. (This latter arrangement is one offering a high-band- 
width reprogramming channel through which, e.g., extensive 
firmware instructions might be transferred to the detector to 
replace instructions earlier stored.) 

By such arrangements, greatly increased detector versatil- 
ity and fijnctionahty can be achieved. 

Conclusion 

Many diverse embodiments are reviewed above — each 

50 with a unique set of features. (Still others are disclosed in the 
assignee's patents incorporated by reference.) This specifica- 
tion should be constmed as explicitly teaching that features 
illustrated in one such embodiment can generally be used in 
other embodiments as well. Thus, for example, a date field 

55 was not particularly discussed in connection with payload 
data for video watermarking. Nor were *'play once" water- 
marks so-considered. The inclusion of a calibration signal 
with (or as part oO the watermark is shown in embodiments of 
the issued patents, but is not belabored in the above-described 

60 embodiments. Likewise with "simple universal codes." The 
pre-stored commerce profile described in one of the forego- 
ing embodiments is equally applicable to other embodiments 
as well. Likewise, the presentation of advertising was dis- 
CTissed in connection with one embodiment but not others, 

65 although it, too, is generally applicable. All of these concepts 
are familiar at Digimarc and are regarded as generally appli- 
cable throughout the woiic expressed in Digimarc 's patent 
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disclosures. Practicality prevents an exhaustive recitation of with reference to at least the information, transmitting at 

each individual permutation and combination. least a portion of the audio ID to a location remote from 

Having described and illustrated the principles of our said device, 

invention with reference to illustrative embodiments, it will 4. A method comprising: 

be apparent that the detailed arrangements can be modified in 5 receiving audio at a device by a microphone; 

arrangement and detail without departing from such prin- providing the audio to a processing system; 

ciples. receiving from the processing system an audio ED decoded 

For example, while reference has been made to various from the audio; 

uses of wireless, it should be understood that such reference obtaining a user ID from a memory in the device; 

does not just cover FM broadcast, and wireless internet net- i o transmitting at least portions of both the audio E) and the 

working and the hke, but also includes other wireless mecha- user ED to a location remote from said device; 

nisms. Examples include cell phones and direct satelUte receiving from the processing system at least two IDs cor- 

broadcast. responding to the audio, one being said audio ID, 

Likewise, while certain embodiments were illustrated with -another being an ID corresponding to an environment in 

a watermark pay load of 100+ bits, in other systems much 15 which the device is located, 

smaller (or sometimes larger) pay loads are desirable— some- 5 . A method comprising : 

times as small as 1-8 bits. receiving electronic signals representing ambient music 

While the foregoing examples have each been illustrated captured with a microphone in a user device; 

with reference to a particular media type (e.g., video, audio, transferring the electronic signals representing the ambimt 

etc.), it will be recognized that the principles of each embodi- 20 music to a multi-purpose electronic processor; 

ment find application with the other media types as well. receiving from the multi-purpose electronic processor data 

Certain of the appliances contemplated above require user derived from the electronic signals; 

interfaces more sophisticated than are presently typical on using said data to obtain information from a database, said 

such devices. The simplicity of the underlying audio appli- information relatmg to the ambient music; 

ance can be preserved, in many instances, by using a palmtop 25 providing at least textual information to a user about the 

computer — coupled by infrared or otherwise — as a tempo- ambient music, said provided textual information being 

rary user interface to tiie appliance. Some of the processing based at least in part on information obtained from the 

capability can hkewise be off-loaded to an ancillary palmtop. database; 

(Pahntop is here meant to refer generally to any pocket-size providing to the user a pointer to an online site, the online 

programmable computing device.) 30 site including third-party reviews of content. 

Unless otherwise stated, it should be understood that the 6. The method of claim 5 in which the textual information 

digital music, video, and imagery contemplated herein is not provided to the user specifies the artist and title of the ambient 

of any particular form or format. Audio, for example, can be music. 

of various forms, both streaming and non-streaming, and of 7. The method of claim 5 in which the textual information 

various formats (e.g. MP3, MP4, MS Audio, Windows Media 35 provides the user an opportunity to have the ambient music, or 

Technologies, RealAudio, *.WAV, MIDI. Csound. Dolby's data related thereto, electronically sent to a destination 

Advanced Audio Codec (AAC), etc. device. 

To provide a comprehensive disclosure without unduly 8. The method of claim 7 that fiirther includes the act of 

lengthening the present specification, applicants incorporate electronically sending the ambient music, or data related 

by reference the patent pubUcations and applications cited 40 thereto, to said destination device. 

herein. 9. The method of claim 7 in which the destination device is 

We claim* distinct from the user device. 

I A method comprising- 10. The method ofclaim 5 in which the textual information 

receivin<y audio at a de^^ce* identifies packaged media on which the music is available. 

discemiSgfromtheaudioapluial-bitaudioIDcomprising . ll- The jnethod of claim 5 in which the user device 

a Digital Object Identifier* mcludes a display, and the textual mformation is presented to 

obtaining information from a memory in the device; ^^^^ ^^^^ display. 

with reference to at least the infomiation, transmitting at . \\:^^ ?f ^^^^ ^ triggered by a user action 

least a portion of the audio ID to a location remote from "^^^^ P^^^l^f a button on the user device, 

said device Jiiet^od of claun 5, tnggered by a voice command 

2. A method comprising: ^^^^ ^^^^ ^ recognition feature of the 

user device 

receiving audio at a device by a microphone ^^^^^ ^^^^ 5 ^^^^ ^^^^ ^ ^^^^ 

discemmg from the audio a plural-bit audio ID; to carry in a user's pocket, 

obtammg a user ID from a memory m the device; 55 15 ^^^^^ ^^^^^^ 5 includes: 

transmittmg at least portions of both the audio ID and the transmitting data from the user device to a remote com- 

user ID to a location remote from said device; and pyter, said data including user/device data relating to at 

discaning at least two IDs from the audio, one being said i^ast one of the following: user name, audio delivery 

audio ID, another being an ID corresponding to an enxa- information, user age, user gender, model of user device, 

ronment in which the device is located. device UID, or user UID; 

3. A method comprising: wherein the text presented to the user is dependent, at least 
receiving audio at a device; in part, on said transmitted user/device data, 
providing the audio to a processing system; 16. The method ofclaim 5 that further includes the act of 
receiving from the processing system an audio ID decoded processing the transferred electronic signals to derive the 

from the audio, tiie audio ID comprises a Digital Object 65 data. 

Identifier, 17. The method ofclaim 16 in which at least some of said 

obtaining information from a memory in the device; processing is performed in the user device. 
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18. The method of claim 16 in which the processing com- 
prises decoding a watermark from the transferred electronic 
signals. 

19. The method of claim 5 in which the user device com- 
prises a cell phone, 

20. The method of claim 5 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

21. A progranuned computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 20. 

22. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-piu*- 
pose electronic processors to perform the acts of claim 20. 

23. The method of claim 5 in which the data comprises an 
identifier. 

24. The method of claim 5 in which the pointer comprises 
one or more URL links. 

25. The method of claim 5 in which the online site provides 
audio associated with the ambient music for a fee. 

26. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 5. 

27. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 5, 

28. The method of claim 5 in which the multi-purpose 
electronic processor is programmed for deriving data from 
the electronic signals. 

29. A method comprising: 

receiving electronic signals representing ambient music 
captured with a microphone m a user device, the .user 
device comprises wireless transmit and receive capabili- 
ties; 

transferring the electronic signals representing the ambient 
music to a mxilti-purpose electronic processor; 

receiving from the multi-purpose electronic processor data 
derived from the electronic signals; 

using said data to obtain information from a database, said 
information relating to the ambient music; 

providing at least textual information to a user about the 
ambient music, said provided textual information being 
based at least in part on information obtaioed from the 
database; 

the wireless device has a store-and- forward capability, 
wherein ambient music can be stored and later identified 
if wireless service is not available at the time when the 
ambient music is received by the microphone. 

30. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 29. 

31. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 29. 

32. The method of claim 29 in which the multi-purpose 
electronic processor is prograromed for deriving data from 
the electronic signals. 

33. The method of claim 29 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

34. One or more computer readable media comprising 
instructions stored thereon to caxise one or more multi-pur- 
pose electronic processors to perform the acts of claim 33. 

35. A programmed computing system storing instructions 
ui memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 33. 
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36. A method comprising : 

receiving electronic signals representing ambient music 

captured with a microphone in a user device; 
transferring the electronic signals representing the ambient 

music to a multi-purpose electronic processor; 
receiving from the multi-purpose electronic processor data 

derived from the electronic signals; 
using said data to obtain information from a database, said 

information relating to the ambient music; 
providing at least textual information to a user about the 

ambient music, said provided textual information being 

based at least in part on information obtained from the 

database; and 

providing to the user a pointer to an online site that pro- 
vides streaming media content associated with the ambi- 
ent music. 

37. The method of claim 36 in which the data comprises an 
identifier. 

38 . The method of claim 36 in which the pointer comprises 
one or more URL links. 

39. The method of claim 36 in which the user device 
comprises a cell phone. 

40. The method of claim 36 in which the user device 
mcludes one or more speakers, and the streaming media con- 
tent is rendered to a iiser via at least the one or more speakers. 

41. The method of claim 36 in which the data is derived 
through decoding a watermark from the electronic signals. 

42. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 36. 

43. One or more computer readable media comprising 
instmctions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 36. 

44. The method of claim 36 in which the multi-purpose 
electronic processor is programmed for deriving data from 
the electronic signals. 

45. The method of claim 36 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

46. A programmed computing system storing instructions 
in memory, said instmctions are executable by said pro- 
grammed computing system to perform the acts of claim 45. 

47. One or more computer readable media comprising 
instmctions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claim 45. 

48. A method comprising: 

receiving electronic signals representing ambient music 

captured with a microphone in a user device; 
transferring the electronic signals representing the ambient 

music to a multi-puipose electronic processor; 
receiving from the multi-purpose electronic processor data 

derived from the electronic signals; 
using said data to obtain information from a database, said 

information relatiog to the ambient music; 
providing at least textual information to a user about the 

ambient music, said provided textual information being 

based at least in part on mformation obtained from the 

database; and 

providing to the user a pointer to dynantic user metadata 
that is associated with the ambient music. 

49. The method of claim 48 in which the dynamic user 
metadata is updatable over time. 

50. The method of claim 49 in which the dynamic user 
metadata comprises artist concert sdieduling. 

51. The method of claim 49 in which the dynamic user 
metadata comprises fan-generated metadata. 
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52. The method of claim 48 in which the data comprises an 
identifier. 

53. The method of claim 48 wherein the pointer comprises 
one or URL links. 

54. The method of claim 48 in which the user device 5 
comprises a cell phone. 

55. The method of claim 48 in which the data is derived 
through decoding a watermark from the electronic signals. 

56. A programmed computing system storing instructions 

in memory, said instructions are executable by said pro- lo 
grammed computing system to perform the acts of claim 48. 

57. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform the acts of claun 48. 
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58. The metliod of claim 48 in which the multi-purpose 
electronic processor is progranamed for deriving data from 
the electronic signals. 

59. The method of claim 48 in which the electronic signals 
representing the ambient music are filtered prior to deriving 
the data. 

60. A programmed computing system storing instructions 
in memory, said instructions are executable by said pro- 
grammed computing system to perform the acts of claim 59, 

61. One or more computer readable media comprising 
instructions stored thereon to cause one or more multi-pur- 
pose electronic processors to perform .the acts of claim 59. 


